✓
Reading for QA
Verification and evals — shipping on proof.
Ship Confidence5 posts
Evals — ship on proof, not vibes.
- 01 · What Are Evals? The gap between 'it worked when I tried it' and engineering has a name: evals. They're repeatable, measured tests of an AI system's output quality — and they're the difference between shipping on confidence and shipping on hope.
- 02 · An Intro to DeepEval You decided evals matter. Now you need to actually write one. Here's DeepEval from zero — what a test case is, how metrics judge it, and how to read your first red bar.
- 03 · A Custom G-Eval Grade with a Claude Subagent Off-the-shelf metrics never fit your domain. Here's the concrete build: a Claude Code subagent that acts as a G-Eval judge, wired into Vitest so a rubric you wrote becomes a pass/fail in CI — with the full agent definition and the test code.
- 04 · How Evals Enable Self-Improving Harnesses Evals don't just catch regressions — they turn a harness into a system that gets better on purpose. Here's the empirical loop that compounds measurement into a flywheel, and why without it every change is a guess.
- 05 · Eval-Driven Development TDD said: write the test first, then the code that passes it. EDD says the same thing one level up — write the eval first, then build the AI system that scores against it. Here's the discipline, and why Meta's just-in-time testing is the same idea arriving from the other direction.