Visualizing reports#
A report can show where a generator differs from real data. The interactive figures use Plotly: hover to inspect values or use the toolbar to save an image. Where a legend is shown, click an entry to show or hide that series.
Start with the real reference#
figure = report.fidelity.plot(kind="scores", metrics=["shape", "trend"])
figure.show()
Dots show the reported estimates: repetition means when uncertainty is enabled for that side, or a single score otherwise. Hover text identifies whether a dot is a point or a mean. Plotting reads retained results without resampling.
In the example below, the generated–real points are shown alongside real–real means and bands, which provide the yardstick. Ten repeated real-real comparisons form the reference percentile bands.
View the source values
| metric | stat | value | ideal | ideal_ci_low | ideal_ci_high |
|---|---|---|---|---|---|
| shape | shape | 0.7042 | 0.9319 | 0.9097 | 0.9516 |
| trend | trend | 0.6967 | 0.8937 | 0.8758 | 0.9138 |
These bands describe variation in aggregate scores across sampled comparisons. They are not confidence intervals for the population expected scores. See statistical methodology for interpretation. Component plots below show descriptive point scores, without column, pair or slice uncertainty estimates.
Find a weak column#
Retain components with details=True when evaluating, then plot the stored
Shape scores:
report.fidelity.plot(kind="columns", metric="shape").show()
View the source values
| column | score |
|---|---|
| visits | 0.8233 |
| spend | 0.7233 |
| tenure | 0.4367 |
| segment | 0.8333 |
The tenure column has the lowest marginal similarity.
This locates a discrepancy; it does not establish its cause. To inspect the data themselves, pass the input tables explicitly:
from tabgen_eval.visualization import plot_column
plot_column(holdout, generated, "tenure", schema=schema).show()
Numeric columns use empirical cumulative distributions; categorical columns use frequency bars. They show the supplied tables using predefined sampling for display, so they need not show the exact row subset used for a report’s draw.
Inspect relationships#
report.fidelity.plot(kind="pairs", metric="trend").show()
View the source values
| column_1 | column_2 | score |
|---|---|---|
| visits | spend | 0.4381 |
| visits | tenure | 0.9957 |
| visits | segment | 0.7100 |
| spend | tenure | 0.8664 |
| spend | segment | 0.7100 |
| tenure | segment | 0.4600 |
Trend compares each column pair. A heatmap makes weak pairs visible even when the aggregate hides them. An undefined entry is distinct from a low score. Compare the actual numeric relationship with a bounded scatter plot:
from tabgen_eval.visualization import plot_column_pair
plot_column_pair(holdout, generated, "visits", "spend", schema=schema).show()
View the source values
Real input: 600 rows. Generated input: 300 rows. The generated spend values were permuted before repeating 75 records.
The inputs show a strong real association and a weaker generated association.
The generated point pile also reflects the repeated record in this example.
This view displays all 600 holdout and 300 generated rows; the report’s metric
comparisons use matched 300-row samples instead.
Scatter plots use at most max_points=3000 rows per table by default; set the
sampling seed to reproduce a view of larger inputs.
Inspect a conditional slice#
Choose a conditioning column and one retained slice statistic:
report.fidelity.plot(kind="slices", metric="cond_shape", column="segment", stat="shape").show()
View the source values
| column | label | reference_count | candidate_count | reference_total | candidate_total | eligible | rare | shape | lower | upper | left_closed | right_closed |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| segment | regional | 94 | 144 | 300 | 300 | True | False | 0.4679 | NaN | NaN | False | False |
| segment | online | 47 | 29 | 300 | 300 | True | False | 0.6356 | NaN | NaN | False | False |
| segment | local | 159 | 127 | 300 | 300 | True | False | 0.7803 | NaN | NaN | False | False |
These scores compare the other columns within each real-defined segment.
Hover for counts and eligibility. They summarize the fixed generated–real
comparison; real–real slice scores are not retained. To inspect frequency
recovery instead, use metric="recovery", stat="recovery".
Compare concentration#
report.fidelity.plot(kind="concentration").show()
View the source values
| radius_id | radius | reference_mean | candidate_mean | mean_ratio | reference_upper | candidate_upper | upper_ratio | reference_n | candidate_n |
|---|---|---|---|---|---|---|---|---|---|
| radii_1e-3 | 0.0010 | 0.0000 | 0.0635 | 6354516.0502 | 0.0000 | 0.2508 | 2.508361e+07 | 300 | 300 |
| q05 | 0.1784 | 0.0503 | 0.0810 | 1.6113 | 0.1070 | 0.2508 | 2.343700e+00 | 300 | 300 |
| q25 | 0.4789 | 0.2449 | 0.2414 | 0.9858 | 0.4348 | 0.3779 | 8.692000e-01 | 300 | 300 |
The fixed small radius exposes the repeated generated record: generated local mass is positive while real local mass is zero. The upper profile is a quantile of local masses, not an uncertainty bound. Larger real-derived radii probe other scales. This figure uses the stored point profiles and actual radii; the scalar concentration gaps have their own real–real comparisons in the report.
PRDC’s four geometric-fidelity scores use the same overview interface:
report.fidelity.plot(kind="scores", metrics=["precision", "recall", "density", "coverage"]).show()
See the PRDC guide for this example’s result.
Report families and available views#
All specialized reports inherit lookup, tables and exports from Report.
Select a family from a mixed report without recomputing its results:
fidelity = report.fidelity
memorization = report.memorization
utility = report.utility
View |
Plot kinds |
|---|---|
|
|
|
|
|
|
PRDC belongs to fidelity because it measures local geometric fidelity. Missing
component tables require a new evaluation with details=True; plotting cannot
recover data that were not retained. Memorization plots show available scalar
summaries, rather than reconstructing distance distributions from quantiles.
Customize, save and reproduce#
Every plotting function returns a Plotly Figure, so normal Plotly controls
apply:
figure = report.fidelity.plot(kind="columns", metric="shape", top_n=10)
figure.update_layout(title="Marginal fidelity by column")
figure.write_html("shape.html", include_plotlyjs=True)
That HTML file includes its plotting library and works offline. Numerical tables and report JSON remain available through the report API.
Run the complete CPU-only example from a source checkout:
python examples/visualization_walkthrough.py --output visualization-demo
It writes seven interactive figures and a JSON report. The files share one
local Plotly JavaScript bundle, and require no network access to open.
Download the example report.
def make_report(holdout, generated):
"""Evaluate once, retaining the component scores used by the figures."""
return evaluate(
EvalData(
test=holdout,
generated=generated,
schema=Schema(numeric=["visits", "spend", "tenure"], categorical=["segment"]),
),
metrics=[
"shape",
"trend",
"cond_shape",
"cond_trend",
"recovery",
"precision",
"recall",
"density",
"coverage",
"concentration",
],
details=True,
device="cpu",
seed=23,
uncertainty={"mode": "reference", "n_replicates": 10},
)