Evals: Proving a Solution Works

⏱ ~7 min read · 🎧 6 min listen · ✎ 3 quizzes · 🧪 ~40 min lab

Demo day goes great. Then the customer's engineering lead asks the only question that matters: "How do you know it works?" — and "we tried a bunch of examples and they looked good" is not an answer a business bets on. This lesson is the course's center of gravity: evals — the discipline of measuring model behavior so you can prove, improve, and protect a solution. It's also the single most cited difference between AI products that ship and ones that stall.²

Why vibes fail

Three mechanisms make eyeballing worthless at scale. Sampling variance (Lesson 01): five good outputs tell you little about the distribution they came from. Unrepresentative tries: you test the inputs you think of; users supply the ones you didn't. And the killer — regressions: Lesson 03 taught you every prompt change moves the whole distribution, so the tweak that fixed ticket #4 may quietly break tickets #1 and #7. Without tests, you'd never accept that in ordinary software. This is ordinary software.

So the reframe: an eval suite is unit tests for model behavior. And exactly as in software engineering, the payoff is iteration speed — Hamel Husain, after autopsying many failed AI products, found the common root cause was precisely the missing evaluation system, because without one every improvement is a guess you can't verify.² Your Lesson 03 lab's pass-rate checker was a tiny eval. Today it grows up.

Anatomy of an eval

Four parts, all of which you can build in an afternoon. A test set: real inputs mirroring the actual task distribution — including edge cases (empty input, hostile input, ambiguous cases). A runner that feeds each input through your system. A grader that scores each output. A score you track over time, per version — the number that answers the engineering lead.¹

Before any of it: define success criteria — Anthropic's docs put this before prompt engineering itself. Good criteria are specific, measurable, achievable, and relevant. Not "the summarizer should be accurate," but "≥90% of summaries preserve all action items, on a 200-ticket held-out set, at under 2¢ each."¹ Notice a criterion like that also settles the Lesson 02 model-tier debate with data.

The three graders

Code-based grading is the workhorse: exact match, string match, schema checks, numeric tolerance. Fastest, cheapest, perfectly reliable — use it for everything that has a checkable answer. Lesson 04 made this easier than it looks: structured outputs turn fuzzy prose into fields that == can grade.¹

Human grading is the gold standard and the bottleneck: flexible, high-quality, slow, expensive. Use humans to bootstrap — label a seed set, define what "good" means, spot-check — not as the per-release grading engine. Anthropic's guidance is blunt: avoid it where possible.¹

LLM-as-judge covers the middle: qualities code can't check and humans can't scale — tone, groundedness, helpfulness. A second model grades each output against a written rubric. Three rules make it trustworthy: give a detailed rubric, force a constrained verdict (correct/incorrect, or 1–5) after brief reasoning, and — before you rely on it — validate the judge against human labels. An unvalidated judge is just vibes with an API bill.¹

Two closing principles from the field. Volume over polish: many automatically-graded cases beat a few hand-graded ones.¹ And evals compound: the same suite that proves v1 works also catches regressions on every prompt change, settles model swaps and price debates with numbers, and — as you'll see next lesson — becomes the yardstick that tells you whether fine-tuning earned its cost.²

🧪 Lab: build the eval, then audit the judge

The lab file is labs/0007-evals.py. Part 1 is a code-graded eval: ten labeled support tickets, two competing prompts, exact-match grading — and a verdict on which prompt wins, with the per-case failures printed. It's your Lesson 03 loop, formalized into a reusable harness.

Part 2 adds an LLM judge for something code can't grade — whether an answer is fully grounded in a provided policy — using a rubric, forced correct/incorrect verdicts, and reasoning before judgment. Part 3 is the step most teams skip: the file ships human labels for every case, so you measure your judge's agreement with humans and decide whether it's trustworthy enough to scale.

You are a senior AI engineer reviewing my work: the output of my evals lab (labs/0007-evals.py) plus a success criterion I wrote for a feature of my choice and my judgment call on whether my LLM judge is trustworthy.

Grade each item as Strong / Adequate / Missing, with one sentence of evidence:
- My success criterion is specific, measurable, achievable and relevant — a number on a defined test set, not an adjective.
- My Part 1 output shows a code-graded comparison of two prompts with per-case failures, and my winner claim is backed by the scores.
- My Part 2 judge uses a written rubric, constrained verdicts, and reasoning before judgment — and I used a different model (or can say why not).
- My Part 3 agreement analysis leads to an explicit trust decision: at what agreement rate I'd rely on this judge, and what I'd do if it fell below.

Be skeptical — challenge my weakest number first. Then ask me 2–3 follow-up questions a customer's engineering lead would ask before accepting my eval as proof. Finish with the single highest-leverage improvement.

My work follows below.

Check yourself — design the measurement

Three teams, three measurement problems. Don't scroll up — reason from the grader trade-offs. Wrong picks stay live.