Read your first report#

Continue with the report returned by the quickstart example. This walkthrough explains its scores and uncertainty bands and follows an aggregate score down to individual columns. For method and field definitions, use the Report API.

The printed summary#

The quickstart produces this summary:

TabGenEval report   (inputs: test=400, generated=200 rows; selected rows per role=100; B=10, 95% score bands)

Fidelity (Distribution Similarity)
  Shape                  mean 0.8543 [0.8219, 0.8733]   real-real mean 0.8973 [0.8619, 0.9303]  finite score replicates G/R 10/10 / 10/10
  Trend                  mean 0.7898 [0.7640, 0.8142]   real-real mean 0.7935 [0.7635, 0.8417]  finite score replicates G/R 10/10 / 10/10

The generated mean averages generated-real comparisons. The real-real mean averages comparisons of two disjoint real samples. Both use 100 rows per side in this example, so the reference gives context at the same sample resolution.

Shape and Trend are higher-is-better similarity scores. Here, Shape is lower than its real-real reference, as is Trend. Read each score in the context of what its metric measures and the available sampling variability.

The quickstart requests ten repetitions for both synthetic-vs.-real and real-vs.-real comparisons. The percentile bands describe variation across comparisons.

Retrieve a result#

Read the Shape result directly:

shape = report["shape"]
print(shape.value)  # Mean generated-real score
print(shape.ideal)  # Mean real-real score
print(shape.n_eff)  # Rows per comparison role

The result lookup and field reference explains qualified names and the other available attributes.

Request real-reference variability only#

Reuse eval_data prepared in the quickstart. To repeat only the real-real comparison, call evaluate with reference uncertainty:

report_with_bands = evaluate(
    eval_data,
    metrics=["shape", "trend"],
    device="cpu",
    seed=42,
    uncertainty={"mode": "reference", "n_replicates": 10},
    details=True,
)

This requests ten real-real comparisons, sampling without replacement within each draw. With these inputs, reference-only mode uses 200 rows per role by default, so its reported scores can differ from the quickstart’s 100-row comparisons. See uncertainty options for other modes, repetition counts and sample resolution.

The same example now gives:

metric value ideal ideal_ci_low ideal_ci_high n_eff
shape 0.8883 0.9262 0.9038 0.9455 200
trend 0.8494 0.8411 0.8194 0.8616 200

ideal_ci_low and ideal_ci_high give the 2.5th and 97.5th percentiles of the finite real-real scores. This band describes variation among sampled comparisons at this resolution. It is not a confidence interval for the expected real-real score \(\mu_n(P)\). Compare the generated score with that reference variability; the report attaches no automatic quality label.

The methodology explains the real-real reference and what the intervals mean.

Inspect individual columns and pairs#

The quickstart uses details=True to retain component scores. Retrieve them from the quickstart report:

columns = report.details("shape")
pairs = report.details("trend")

Its Shape table is:

column  score
   age   0.85
income   0.86
region   0.91

The age column has the lowest marginal similarity in this example. Component tables help locate differences in the first generated-real repetition. They are not averages over repetitions and have no per-column uncertainty bands; their scope is recorded in the detail metadata. For your own evaluations, request details=True when you need these tables.

Visualize stored results#

Plot the same column scores through the report’s fidelity view:

report.fidelity.plot(kind="columns", metric="shape").show()

This uses stored results without recomputing the metric. Continue with the visual walkthrough to investigate weak columns, relationships and subpopulations. The family-view reference explains how to select fidelity, memorization and utility results from a mixed report.

Export and retain context#

Save the report with its reference interval:

report_with_bands.to_json("evaluation.json")

JSON keeps the scores, schema, resolved options and execution diagnostics. Add include_replicates=True to retain the ordered scores used to form the bands. The export reference covers other formats and exact serialization behavior.

Download the example JSON report. For undefined scores or unexpected diagnostics, see Troubleshooting.