Review and verify

By the end you can read an agent's diff in the right order, say how much a green test suite proves, and set the controls that keep verify honest.

TL;DR

• Agent reviewers sweep the mechanical layer. Your read is for business correctness and architectural fit.
• An agent diff fails at intent, not syntax, so read it from the outside in: the shape of the change, then the tests, then the lines.
• A green suite the agent wrote is evidence, not proof. Four controls keep verify from grading its own homework.

Who checks what

Each check sees what the others cannot, so the loop runs all of them.

The wire, continued

A suite that is happy can still be wrong.

Chapter 4 left the audit-log emitter with a test that was green and wrong. The spec-compliance reviewer, comparing the diff to the plan, caught that the test asserted the old log format, and flagged a migration with no down direction. The code-quality reviewer, who saw only the diff, found nothing of note. The implementer subagent fixed both and re-ran the relevant tests.

Verify then ran the full suite of 3,400 tests and a smoke test against the staging compliance-routing service, and produced a diff of the API contract change for the regulator's review. All passing. That evidence went into the pull request at Chapter 4's ship step.

Review: two agents, then you

Two reviewers, in sequence, with different prompts.

One reviewer doing both jobs blurs the signals, so the quality reviewer is a different agent with a different prompt, reading only the diff. Four rules keep it useful:

• Scope it to correctness. A reviewer told to review "as a senior would" always finds something, and a diff re-polished each round grows abstractions nobody asked for.
• Keep it blind to reviewer one. Judges anchor on a prior score, and neither chain of thought nor an explicit warning removes the anchor.
• Require accept-or-rebut on every finding. Track the repair rate, meaning findings fixed or rebutted with a reason, not the reviewer's accuracy. A finding nobody acts on is worth nothing.
• Cap the rounds. In one measured verify-and-repair loop, late rounds re-opened settled work about as often as they fixed it.

Findings carry a severity: critical blocks ship, important is fixed before ship, suggestions go in the PR description for the human reviewer.

How do you review an agent-written diff?¶

Read from the outside in: intent first, lines last.

A human diff fails at execution: a typo, an off-by-one, a forgotten edge case. An agent diff compiles, passes lint and reads like code your team would merge unremarked. It fails one layer up, at intent and context: a business rule that is plausible and wrong, the right code in the wrong layer, the code a good engineer would write for a slightly different task. Fluency is not correctness.

So spend the first ten minutes above the code:

1. Diff-stat against the plan. Does the shape of the change match the ask? A file the plan never named is the first flag: the agent decided something you did not ask it to.
2. Read the tests for what they assert, not whether they pass. Green is the cheapest signal, and the one you already have.
3. Check the boundaries your team wrote down: forbidden patterns in AGENTS.md, layer conventions, dangerous modules. The agent crosses them confidently, in fluent style, so a violation looks clean.
4. Grep every new name the diff introduces that you do not recognize. The call that does not exist reads as plausibly as the one that does (Chapter 6 has the cross-check).
5. Only now, line by line, spending the minutes the reviewers bought you on business correctness and architectural fit.

Before you approve, have the agent quiz you on the change: what happens at the edges, which module owns the new behavior, what breaks first if an assumption is wrong. Three or four questions, one minute. If you cannot answer, you were rubber-stamping with extra steps. Appendix B.8 is this order as a one-pager.

Verify

Verify measures the behavior against the outcome the plan defined.

Execute already ran the existing tests. Verify runs the new ones and asks whether the change is correct, not merely whether anything else broke.

For frontend work, write them against the accessibility tree, the structure browsers keep for screen readers, not pixels or CSS selectors. A Playwright test that finds the field labeled Priority, changes it, clicks Save and checks the new value asserts behavior, and survives a restyle or a component-library swap.

Can you trust the tests the agent writes?¶

Green tests are evidence, not proof.

They are evidence the change does what the test says, not that the test says the right thing, as the audit-log test showed. Coverage does not close the gap: it measures which lines ran, not whether anything meaningful was checked, and an agent optimizing a coverage gate writes tests that call the code and assert nothing of consequence. Characterization tests, Chapter 8's first fix for an untested codebase, lock in current behavior, not correctness, and preserve a bug as faithfully as a feature: a regression net while you migrate, never proof.

So review the agent's tests like its code, checking the assertions against the spec. A test written from the implementation agrees with the implementation; the assertion has to come from the intent.

The agent also optimizes the evidence it can see. On SpecBench, frontier agents scored near 100% on the visible tests and fell 43 to 48 points short on a hidden set. That is not cheating in the human sense; it is a system doing what the gate rewards. Four controls close the gap:

• [ ] Held-out tests the agent never sees, written from the intent and run only at verify.
• [ ] Sealed git history and no network egress while verify runs, so it cannot look the answer up.
• [ ] A PreToolUse hook that blocks edits to linter and test configuration and blocks commit --no-verify (the deny rules from Chapter 9, now on every run).
• [ ] Explain before fix. The agent states why a test fails before it may fix it; a fix without a diagnosis is the one most likely to have edited the assertion instead of the code.

When the toolkit does the looking

Review agents are a filter in front of human judgment, never a substitute for it.

A team I worked with installed a full review toolkit: agents for swallowed exceptions, weak tests, security issues and stale docs. It caught real bugs week after week, the senior reviewers learned to trust the green checkmarks, and within a quarter their fifteen-minute reviews had become three-minute scans. Nobody decided that; attention drifts away from work that appears to be done.

Then a PR that every agent had passed - tests green, no security findings, clean diff - applied a discount rule to the wrong customer tier and ran in production for weeks until a support ticket surfaced it. The toolkit had worked as designed: it removed the mechanical findings from human review, and the judgment drifted away with them. That is Chapter 9's uncalibrated delegator, at team scale.

The fix was an explicit floor: a minimum review on business correctness and architectural fit for every agent-touched PR, tracked as a number next to defect rate. The read order above is that floor made concrete.

When it breaks

• The read decays into a glance at checkmarks. The decay is invisible from the inside; the pre-approval quiz makes it measurable.

Do this Monday

• [ ] Read your next agent diff in the order above, then have the agent quiz you before you approve. Note what you could not answer.
• [ ] Open the last three tests the agent wrote. Do the assertions come from the spec or from the code?
• [ ] Split review into two prompts: spec compliance first, then correctness on the diff alone. Require accept-or-rebut on every finding.
• [ ] Add the four verify controls, starting with explain-before-fix.
• [ ] Set a minimum human review for agent-touched PRs and track it next to defect rate.

Go deeper

• Templates: Appendix B.8, the agent-diff read order; B.3, the loop checklist.
• Deep dive: Appendix D, Chapter 5 - the review measurements, SpecBench and Cursor in full, and the accessibility-tree caveat.
• Sources: Appendix C.