A green check from an AI reviewer is not the same thing as safe code. Anthropic’s Claude Code team has a better answer to the flood of agent-written changes: put several checks around the work, then spend human attention according to the risk of the change.

Anthropic’s layered code review process, from local verification through staged release and rollback

In a post published October 15, the team says its merged pull requests more than doubled between June and September while the share later reverted fell by more than 40%. That is an encouraging result, not proof that the workflow caused the change. The teams, codebases, and workload may differ from yours. Still, the details make a useful pilot plan for anyone whose coding agents are opening PRs faster than colleagues can inspect them.

The interesting shift is the question the team asks. Instead of treating “a human read every line” as a quality metric, ask what could fail, which check is best suited to catch it, and how quickly you can undo the damage if it slips through. That produces a layered loop, not a ceremonial approval checkbox.

The review loop to copy

Start before the pull request exists. Ask the coding agent to run something that can prove its work: the relevant tests, a build, a screenshot comparison for a UI change, or a small end-to-end check. Ask it to show the output. “Tests passed” is a claim; the command and its result are evidence. Then run a fresh-context review of the diff. The reviewer should see the task and the changed code, but should not inherit the authoring conversation’s assumptions.

When the same defect appears repeatedly, fix the process that keeps producing it. Add a test or lint rule if the failure is deterministic. Put project-wide conventions in CLAUDE.md, and review-specific expectations in REVIEW.md. Anthropic’s guidance is to keep the latter focused: define what counts as a serious finding for this repository, cap style nits, name paths to ignore, and require a file-and-line explanation. A review prompt that says “look for bugs” is too vague to tune when it gets noisy.

Once the PR opens, put an AI reviewer in front of a human, not in place of one. Have it compare the change with the stated intent and surrounding code, then require the author to fix or explicitly resolve each finding. Anthropic’s managed Code Review runs several reviewers in parallel and verifies candidate findings before posting them. The documented average is $15–25 per review and about 20 minutes, with cost varying by PR size and complexity. That makes an automatic review on every push an expensive default for some teams. Start with a manual trigger or a review-ready PR, and watch the actual bill before expanding coverage.

Keep deterministic checks separate. Require tests, type checks, lint, and security scans through CI. Make security findings merge-blocking only when the team can distinguish a real issue from noise and provide a clear resolution path. The managed Claude review check itself is neutral by default; its comments do not block merges. Do not mistake a posted AI comment for a security gate.

A small rollout can look like this:

  • Pick one repository and one class of agent-generated changes.
  • Require the agent to attach test or build evidence before it opens a PR.
  • Add a fresh-context AI review and a short REVIEW.md with repo-specific failure modes.
  • Keep CI and security scans as independent required checks.
  • Make the author resolve every review finding, then measure noise and missed issues.
  • Expand only after the first few weeks show useful findings without a rise in reverts or incidents.

For a first REVIEW.md, write a handful of statements that can be checked against the diff. For example: “Flag a new API route without an integration test”; “Treat a migration that drops a column as Important”; “Ignore generated files under src/generated/”; “Every finding must name the changed file and line and explain a reproducible failure.” Those are starter examples, not universal rules. Ask the code owners which failures would hurt their service, then write those down instead.

Where humans still decide

Anthropic’s own process scales approval by the change, not by the author. Its examples allow low-risk docs or additive changes behind a flag to proceed without a required human approval once automated checks pass. Ordinary product changes get one person. Sensitive paths and the highest-risk work get two. Auth, billing, release workflows, migrations, and infrastructure are sensible candidates for a team’s always-human list, but each codebase should set its own boundaries.

A practical first policy is stricter than the one you hope to reach. Keep human approval for every PR while you run the pilot. Label changes low, medium, or high risk using changed paths and consequences, not whether an agent or a person wrote them. When evidence is strong, consider removing a required approval for a narrow low-risk category. Never let the bot count as one of the required humans, and require approval again if a new commit changes what was reviewed.

The other half is measuring the cost of the policy. Track reviewer wait time and human approvals per PR, but pair those speed numbers with revert and incident rates. Anthropic says it watches both the median and 95th-percentile approval count, along with human comments and time waiting for review. If review time falls while reverts climb, you have moved the cost downstream, not removed it. Their own post says they still lack a circuit breaker that tightens approvals when those guardrail metrics worsen. That caveat matters more than the polished diagram.

When something slips through, add it to a small regression set: the original change, the failure, and the fix. Re-run the reviewer against those examples whenever you change its prompt, model, or tools. A reviewer that never gets tested against the bugs your team actually shipped is just another opinionated bot.

A useful related read is why coding agents still miss reproduction tests, which explains why passing an agent’s own test report is not enough evidence by itself.

You do not need Anthropic’s scale to borrow the useful part. Put proof before PR, automate repeatable checks, keep humans for decisions with real consequences, and make rollback part of the design. Start on one repo. The metric to beat is not “fewer approvals”; it is less human time per change without more incidents or reverts.

Sources