Learn · Developer

Evals: Testing Software That Rolls Dice

You cannot unit-test a model. You can do something better: measure it.

Traditional testing rests on a comforting fact: the same input produces the same output, so one passing assertion settles the matter. Models broke that bargain. The same prompt produces different phrasings, occasionally different substance, and a prompt 'fix' that improves the case in front of you can quietly break five cases you were not looking at. The discipline that replaces unit-test certainty is the eval: run the system against a collection of representative cases, score the results, and watch the number. Less satisfying than green checkmarks; far more honest.

The golden set is the asset

An eval is only as good as its cases, and the useful cases come from reality, not imagination:

Seed from production. The [input-output logs you kept](/build/prompting-for-programs/) are a sampling of what your system actually faces. Pull the typical, the weird, and the long-tail from there.
Every failure becomes a case. A user hits a bad output; the input joins the set, permanently. This is the compounding loop — over months, your eval set becomes a fossil record of every way your system has failed, and nothing regresses silently to a failure you have already paid for once.
Include the boring and the hostile. Ordinary cases that must keep working, empty and malformed inputs, and — for [agents](/build/building-agents/) — inputs containing injection attempts. The eval set is where 'we handle that' becomes checkable.
Small and real beats large and synthetic. Fifty genuine cases you understand outperform five thousand generated ones you have never read. Grow it; do not inflate it.

Grading: three tiers, cheapest first

Code-graded checks carry more weight than newcomers expect. Did the output parse against the schema? Is the category from the allowed list? Does the extracted date appear in the source document? Does the answer contain the required disclaimer? None of this needs judgment, all of it catches real failures, and it runs in milliseconds. Exhaust this tier before reaching for anything fancier.

Model-graded checks — using a model to judge outputs against a rubric — are how you scale judgments code cannot make: is this summary faithful, is this answer grounded in the provided passages, is the tone right. They work, with disciplines attached: write the rubric as concrete criteria rather than 'rate the quality'; prefer pairwise or pass-fail judgments over ten-point scores, which judges use inconsistently; and calibrate the judge against human labels on a sample before trusting it — an unvalidated judge is just a second opinion of unknown quality. Judges also carry known biases, favoring longer and more confident answers, so spot-check them periodically rather than once.

Human review is the expensive tier and the ground truth. Spend it where it compounds: labeling the calibration sample, adjudicating cases where judge and expectations disagree, and periodically reading raw transcripts — which surfaces whole categories of weirdness no metric was watching for.

Wire it into the loop

The mechanics only matter if they gate changes. The habit that separates teams with evals from teams with anxiety: no prompt, model, or retrieval change ships without a before-and-after run. Prompt edits feel trivially safe and are empirically the opposite — a clarifying sentence can shift behavior across the whole distribution. The eval turns 'this feels better' into 'this moved pass rate from 86 to 91, and here are the three cases that got worse.' Those three cases matter as much as the average: aggregate scores hide concentrated damage, so always diff at the case level.

Model upgrades get the same treatment. Providers retire and improve models on their schedule, not yours; an eval suite converts each upgrade from a leap of faith into an afternoon's measured decision — often revealing, pleasantly, that [a cheaper model now clears your bar](/build/choosing-models-costs/), which is the subject of the final guide in this track.

Keep the ambition honest

Start embarrassingly small: twenty cases and a schema check beat a quarter-long evaluation initiative that never ships. Let production failures grow the set. Resist the metric-zoo — one or two numbers a team actually watches outperform a dashboard of twelve nobody interprets. The test of a good eval practice is behavioral, not technical: when someone proposes a prompt change, does anyone ask what the eval said? Once the answer is reflexively yes, you have the thing evals exist to provide — the ability to change AI systems on purpose.

Common questions

What is an eval for an AI application?

A collection of representative test cases run against your system with scored results — the AI equivalent of a test suite, built for software whose outputs vary. Instead of asserting one exact answer, you measure pass rates across real cases and compare before and after every prompt, model, or retrieval change.

Where do eval test cases come from?

Production, overwhelmingly: sample your logged inputs for typical and unusual cases, and add every real failure to the set permanently so it can never silently regress. Fifty genuine, understood cases beat thousands of synthetic ones. Include boring must-keep-working inputs, malformed inputs, and hostile ones.

Is using an LLM as a judge reliable?

Usable with discipline, not blindly: give the judge concrete rubric criteria rather than open-ended quality ratings, prefer pass-fail or pairwise judgments over numeric scales, and calibrate it against human labels on a sample before trusting it. Judges skew toward longer, more confident answers, so recheck periodically.

Why did my prompt change break things it should not affect?

Prompt edits shift model behavior across the whole input distribution, not just the case you were fixing — a clarifying sentence can alter unrelated outputs. This is why eval runs before and after every change, with case-level diffs rather than averages alone, are the core habit of teams that ship AI reliably.