Read your first report#
Continue with the report returned by the quickstart example.
This walkthrough explains its scores and uncertainty bands and
follows an aggregate score down to individual columns. For method and field
definitions, use the Report API.
The printed summary#
The quickstart produces this summary:
TabGenEval report (inputs: test=400, generated=200 rows; selected rows per role=100; B=10, 95% score bands)
Fidelity (Distribution Similarity)
Shape mean 0.8543 [0.8219, 0.8733] real-real mean 0.8973 [0.8619, 0.9303] finite score replicates G/R 10/10 / 10/10
Trend mean 0.7898 [0.7640, 0.8142] real-real mean 0.7935 [0.7635, 0.8417] finite score replicates G/R 10/10 / 10/10
The generated mean averages generated-real comparisons. The real-real mean
averages comparisons of two disjoint real samples. Both use 100 rows per side in this example,
so the reference gives context at the same sample resolution.
Shape and Trend are higher-is-better similarity scores. Here, Shape is lower than its real-real reference, as is Trend. Read each score in the context of what its metric measures and the available sampling variability.
The quickstart requests ten repetitions for both synthetic-vs.-real and real-vs.-real comparisons. The percentile bands describe variation across comparisons.
Retrieve a result#
Read the Shape result directly:
shape = report["shape"]
print(shape.value) # Mean generated-real score
print(shape.ideal) # Mean real-real score
print(shape.n_eff) # Rows per comparison role
The result lookup and field reference explains qualified names and the other available attributes.
Request real-reference variability only#
Reuse eval_data prepared in the quickstart. To repeat only the real-real
comparison, call evaluate with reference uncertainty:
report_with_bands = evaluate(
eval_data,
metrics=["shape", "trend"],
device="cpu",
seed=42,
uncertainty={"mode": "reference", "n_replicates": 10},
details=True,
)
This requests ten real-real comparisons, sampling without replacement within each draw. With these inputs, reference-only mode uses 200 rows per role by default, so its reported scores can differ from the quickstart’s 100-row comparisons. See uncertainty options for other modes, repetition counts and sample resolution.
The same example now gives:
| metric | value | ideal | ideal_ci_low | ideal_ci_high | n_eff |
|---|---|---|---|---|---|
| shape | 0.8883 | 0.9262 | 0.9038 | 0.9455 | 200 |
| trend | 0.8494 | 0.8411 | 0.8194 | 0.8616 | 200 |
ideal_ci_low and ideal_ci_high give the
2.5th and 97.5th percentiles of the finite real-real scores. This band describes
variation among sampled comparisons at this resolution. It is not a confidence
interval for the expected real-real score \(\mu_n(P)\). Compare the generated
score with that reference variability; the report attaches no automatic quality
label.
The methodology explains the real-real reference and what the intervals mean.
Inspect individual columns and pairs#
The quickstart uses details=True to retain component scores. Retrieve them
from the quickstart report:
columns = report.details("shape")
pairs = report.details("trend")
Its Shape table is:
column score
age 0.85
income 0.86
region 0.91
The age column has the lowest marginal similarity in this example. Component
tables help locate differences in the first generated-real repetition. They
are not averages over repetitions and have no per-column uncertainty bands;
their scope is recorded in the detail metadata.
For your own evaluations, request details=True when you need these tables.
Visualize stored results#
Plot the same column scores through the report’s fidelity view:
report.fidelity.plot(kind="columns", metric="shape").show()
This uses stored results without recomputing the metric. Continue with the visual walkthrough to investigate weak columns, relationships and subpopulations. The family-view reference explains how to select fidelity, memorization and utility results from a mixed report.
Export and retain context#
Save the report with its reference interval:
report_with_bands.to_json("evaluation.json")
JSON keeps the scores, schema, resolved options and execution diagnostics.
Add include_replicates=True to retain the ordered scores used to form the
bands. The export reference covers
other formats and exact serialization behavior.
Download the example JSON report.
For undefined scores or unexpected diagnostics, see
Troubleshooting.