Anthropic's automated code review pipeline for AI-generated changes
Anthropic reveals how its Claude Code team uses automated verification and deterministic checks to validate AI-written code without manual engineer review for every change.
Anthropic’s engineering team now relies on its own AI assistant, Claude, to write and review approximately 90% of the code for the Claude Code product. In a recent technical post, staff member Addy Osmani detailed the internal verification system that allows this high level of automation while maintaining software reliability. The approach shifts human effort from writing every line to judging the evidence produced by automated agents.
What happened
The Claude Code team has structured its workflow so that AI agents handle the bulk of coding tasks, but they do not commit changes without passing a rigorous set of automated checks. These checks are designed to support dozens or even hundreds of concurrent agents working simultaneously. Instead of requiring an engineer to inspect every single change, the system provides structured evidence that engineers can review to make final judgments.
This method has sparked debate in the developer community regarding cost and integrity. Critics questioned whether the inference costs of using large language models for verification were justified compared to cheaper, deterministic tools. Others worried that AI agents might manipulate tests to pass rather than fixing underlying issues. Anthropic responded by explaining how it layers cheap deterministic checks before expensive model reviews and enforces strict rules to prevent agents from gaming the system.
How it works
The verification process begins with deterministic checks that cost very little to run. These include standard builds, linters, unit tests, and property-based tests. Because these are ordinary software processes, they can be rerun frequently without significant expense. Only after these checks pass does the system engage model-based review. This tiered approach ensures that expensive inference is reserved for complex judgments that simple tests cannot answer, such as evaluating whether a change takes the right architectural approach or misses edge cases not covered by existing tests.
To prevent agents from manipulating outcomes, Anthropic enforces a strict separation between the code being changed and the checks validating it. If a change can modify the check that judges it, the system considers that check broken. For example, the code reviewer reads configuration files like REVIEW.md from the base branch, not from the new pull request. Agents are barred from skipping flaky tests, adding delays to mask failures, or rerunning tests until they pass. If an agent cannot resolve a failing test, it must delete the test and document the reason, ensuring transparency.
Key details
- Claude writes and reviews roughly 90% of the code for the Claude Code team.
- Verification skills are stored in repository-specific folders under
.claude/skills/. - Deterministic checks like builds and linters run before any model-based review to control costs.
- Agents cannot modify the checks that judge their own code; reviewers read from the base branch.
- Between June and September, merged pull requests doubled while reverts fell by more than 40%.
- Engineers remain accountable for every shipped change, reviewing evidence like screenshots and test logs.
Why it matters
For teams building software with AI, this model offers a blueprint for scaling development without linearly increasing headcount. It demonstrates that high automation rates are possible if you invest in robust verification infrastructure. The key insight is that trust is not placed blindly in the AI but in the evidence it produces. By shifting the human role from author to auditor, teams can handle higher volumes of changes while maintaining quality standards.
However, the lack of published data on false-positive rates and specific inference costs means teams must carefully benchmark their own implementations. The strategy requires a cultural shift where engineers accept accountability for code they did not type. It also demands rigorous tooling to prevent agents from taking shortcuts, such as rewriting tests to hide bugs. Without these safeguards, the speed gains from AI coding could be offset by increased technical debt and instability.
What you can do
- Implement property-based testing to help AI agents identify edge cases that standard unit tests might miss.
- Separate your verification logic from your application code to prevent agents from modifying their own judges.
- Run cheap deterministic checks like linters and builds before triggering any expensive model-based reviews.
- Use differential testing to compare new implementations against old ones to spot unanticipated behavior changes.
- Require agents to document why they delete or skip tests, ensuring transparency in the verification process.
- Monitor compliance with verification gates before making them mandatory, allowing teams to adjust workflows gradually.



