Lean on the Tools


You spend a year learning to conduct — from soloist to a person in front of a team of agents, multiplying your intent across ten parallel threads — and your reward is a brand-new problem that’s entirely your own fault. Every one of those agents produces output. All of it flows back to exactly one place. You.

You didn’t remove the bottleneck. You moved it, straight onto your own desk, and made it worse.

That’s the trap nobody warns you about when they sell you on leverage. Directing one agent, you could read everything it wrote — the volume was human-sized. Direct ten and the math turns on you fast: ten streams of confident, plausible, occasionally-wrong output, and one pair of eyes that has to decide which parts to trust. The typing got cheap. The reading didn’t. And in an agentic workflow the reading is the whole job — the review is the work now, not the overhead around it.

So the constraint stops being how fast the work gets produced. It becomes how fast you can verify it. Your judgment is the scarce resource, and you’ve just pointed a firehose at it.

Why “review harder” is a losing move

The instinct is to grind. More agents, more output, so read faster, read longer, stay up later. Muscle through it.

It doesn’t scale, for two reasons, and they compound.

The first is arithmetic. There is a ceiling on how much output a human can meaningfully review in a day, and it is much lower than the ceiling on how much ten agents can produce. You will hit yours by mid-morning. Past that point you’re not reviewing — you’re skimming, then you’re rubber-stamping, then you’re just clicking approve and hoping. The bottleneck doesn’t go away when you push through it. It just quietly converts from review into unreviewed output shipped with your name on it.

The second is worse, because it’s invisible. When you’re drowning, the muscle that fails first is the one that catches the plausible-but-wrong answer — the subtle bug wearing a confident face. That’s the exact failure mode the series opener is about: point these tools at someone who ships without verifying and they pass unverified, half-understood output downstream at machine speed. Except now that someone is you, on hour nine, because you tried to out-read a firehose. You start shipping tech debt at a rate no human team could have produced on its own — and the worst of it is the context debt, the bad assumption that fans out through the shared harness into every future conversation. Reviewing harder doesn’t prevent that. It’s the thing that causes it.

You can’t fix a scaling problem with more effort. Effort is the thing that doesn’t scale. That’s what makes it the bottleneck.

Lean on the tools

Here’s the move, and it’s the same move a good conductor already made once: you already learned not to play every instrument yourself. Now learn not to verify every note yourself either. Build a layer of tooling that does the first pass of review before your eyes ever touch the output — so your judgment lands only where a machine genuinely can’t reach.

Three tools carry most of the weight.

Specs are the contract that makes output checkable at all. You cannot verify against a standard that doesn’t exist. A vague ask — “clean up onboarding” — produces output you have to eyeball line by line, because “correct” lives only in your head and the machine can’t be graded against a vibe. A real spec turns the fuzzy intent into something with edges: these inputs, that behavior, this is done. That’s the upfront work that makes everything downstream automatically checkable. It’s why the anatomy of a spec is the foundation, not the paperwork — a spec you write and walk away from is a New Year’s resolution, but a spec written to be tested against is a contract the tools can enforce for you.

Evals are automated judgment, running at the scale your eyes can’t. This is the load-bearing one, so let’s define it before I build on it.

eval [ee·val] · noun

Everyday — an assessment or appraisal — a judgment of how good something is.

In agentic development — an automated, repeatable check that grades an AI system's output against a definition of "good," so a machine does the first pass and you don't have to eyeball every result. Judgment, encoded once and run a thousand times.

An eval is you, bottled. You form the judgment once — what “good” means for this task — write it down as a check, and then it runs on every output forever, at machine speed, while you sleep. Ten agents produce ten results; the eval suite reads all ten and hands you a verdict before you’ve had coffee. Evals are the unit tests of the LLM era — and the toolchain for building them is real and boring in the good way, whether you’re wiring up DeepEval or running the whole loop eval-first so the grade drives the work. The point isn’t the framework. The point is that your judgment stops being a thing you spend per-output and becomes a thing you invested once and now collect interest on.

End-to-end tests are the safety net under the whole act. Specs and evals grade whether the pieces are right; the e2e test grades whether the thing actually works when you run it for real — the same way twice, not just the one time you tried it and it happened to pass. It’s the last line, the “it worked when I tried it” lie caught before it reaches anyone else. When you’re conducting ten threads, the e2e suite is what lets you merge without personally re-testing the world.

None of these is exotic. That’s the point. The leverage isn’t in any one tool — it’s in the fact that together they read the output first, so by the time anything reaches your desk, most of it has already been graded.

Move faster, review lighter

Here’s what actually changes when the tooling does the first pass.

The tools handle the bulk — the boilerplate, the regressions, the ten-out-of-ten “yep, this matches the spec.” What survives to your desk is the short list: the exceptions the eval flagged, the genuinely novel call no rubric anticipated, the place where the spec itself is wrong and only a human would notice. That’s where your judgment belongs. Not spread thin across every line of every output — concentrated on the handful of decisions that are actually hard.

So you review lighter, and that’s not a lowering of standards — it’s the opposite. You’re not reviewing less carefully. You’re reviewing the right things carefully and letting the machine handle the rest. The firehose still runs at full pressure. But now it hits a filter before it hits you, and what lands is a trickle of the genuinely-worth-your-attention. Faster throughput, lighter human load, higher standard — all at once, because the review that scales is the review you encoded measurably instead of doing by hand. Trust but verify — except “verify” is now a suite that runs itself, and you’re the last check, not the only one.

The catch, and it’s a big one

Now the discipline, because a tool this good is a tool this dangerous.

An eval only helps if the standard it encodes is right. A test only helps if it tests the thing that matters. A spec only helps if it captures the actual intent. Get any of those wrong and you haven’t built a safety net — you’ve built a machine that certifies garbage as gospel, at scale, with a green checkmark. Garbage in, gospel out. The output passes every check and every check was measuring the wrong thing, and now you trust it more than the unchecked version because it has a passing grade stapled to it. That’s a worse failure than no tooling at all, because it launders bad judgment into false confidence.

Which is exactly why this post is number three and not number one. Leaning on the tools only works if your intent was good to begin with — if you formed real intent worth encoding and learned to conduct instead of grinding out every note yourself. The tools don’t replace judgment. They amplify it — and amplifiers are indifferent to what you feed them. Point good intent through them and you review lighter and ship faster. Point sloppy intent through them and you industrialize your own mistakes.

The tooling is leverage on your judgment, not a substitute for it. First you have to have the judgment. Then you build the machine that spends it a thousand times a day so you don’t have to.

So here’s what I keep chewing on: when a task sails through your whole suite — spec matched, evals green, e2e passing — do you actually read the output anymore? And if you’re honest that you don’t — how sure are you that the checks are testing the thing that would’ve made you wince?


This is Part 3 of Leadership in the Agentic Era. Coming next:

  • 4 · Measure the Right Thing — the scrum metrics that lie in an agentic world, and what frontier teams track instead.
  • 5 · The Incongruent Org — what breaks when half your team moves up the stack and half doesn’t.
  • 6 · Leading the Change — empathy, foundations, and a shared language for the transition.

Comments