A correct answer can
still be incomplete.
Explore two small checks that make AI-assisted systems easier to trust: complete retrieval and reconciled migrations. Every result below is computed from visible synthetic data.
Find every match
A ranked search can return three valid records while silently missing four. Compare the returned IDs with an independently defined expected set.
Paste IDs from a candidate output to check omissions and unsupported records.
This is a deterministic evaluation demo. No model, embeddings, private data, or external APIs are involved. Top 3 is an illustrative failure, not a measured model result.
Evaluation
Inspect the source records and expected sets
Expected IDs are explicit fixtures, separate from the retrieval implementation. Real evaluation sets need domain review, representative queries, and versioning.
Catch the regression before release
Run the same five cases against both strategies. A change can preserve precision while breaking completeness.
Prove the cutover
Row counts alone are weak evidence. This example checks stable IDs, duplicates, money totals, and field-level changes between a legacy export and a candidate database export.
Synthetic order data only. Amounts use integer cents. Nothing is written to a database. A real cutover also needs foreign-key checks, business invariants, access-control tests, a freeze or delta-sync plan, and rollback rehearsal.
Reconciliation report
Compare both exports
| Order ID | Source cents | Candidate cents | Source customer | Candidate customer |
|---|