AI agents

How the Claude Code team scales AI verification loops

The Claude Code team shares how they use end-to-end, property, and differential tests to verify agent-generated code at scale.

A robotic arm inspecting a circuit board with verification symbols.
Illustration generated for this article

The Claude Code team at Anthropic now has Claude write and review approximately ninety percent of its codebase. In a November 2026 blog post, the team detailed how they use structured verification loops to maintain quality while scaling agent usage. This approach allows them to run dozens or hundreds of agents in parallel by automating the evidence-gathering process that usually requires human oversight.

What happened

At this scale, manual checking becomes a bottleneck. The team found that without automated verification, engineers could only manage one or two agents at a time because every output required personal inspection. By equipping agents with the ability to verify their own work against specific rubrics, they unlocked massive parallelism. Internal testing showed that using a separate grader to check structured file generation improved task success by up to ten points, with the most significant gains appearing on the hardest problems.

The core insight is that verification tasks, which were once expensive for humans, are now cheap for agents. Agents can quickly generate tests, benchmarks, and specifications. Once created, these artifacts run as standard software, allowing frequent execution without consuming additional tokens. The team closes the loop by using test results to update agent rules, such as adding explicit constraints to configuration files when an agent repeatedly fails a specific check.

How it works

The team prioritizes three specific types of tests to ensure robustness. First, they favor end-to-end (E2E) tests over unit tests because E2E tests describe real user flows and are difficult to fool. While unit tests might pass even if a feature is broken, E2E tests provide definitive done criteria and concrete evidence of failure, such as screenshots. Agents use tools like CDP or simulators to interact with the running system, guided by feature maps that list expected paths and results.

Second, they use property tests to catch edge cases that humans and agents often miss. Instead of testing specific inputs, property tests define invariants that must always hold true, such as round-trip encoding consistency or non-negative account balances. Libraries like fast-check or Hypothesis generate thousands of random inputs to try and break these rules, providing an unbiased check on agent-generated code. Third, for large-scale migrations, they employ differential tests. These compare the output of the old and new systems using the same inputs, flagging any deviations as potential bugs.

Key details

  • Verification loops allow teams to scale from managing one or two agents to dozens or hundreds in parallel.
  • End-to-end tests are preferred because they validate real user flows and provide concrete failure evidence.
  • Property tests use random input generation to find edge cases that bias-prone agents might overlook.
  • Differential tests compare old and new system outputs to detect regressions during migrations.
  • Agents attempt to fix failing checks for three rounds before escalating the issue to a human reviewer.
  • Verification rules are stored in branches that agents cannot modify, preventing them from altering their own grading criteria.

Why it matters

For engineering leaders and developers building with AI agents, this shift changes the primary job description from reading every diff to defining what "done" looks like. By investing in strong verification layers, teams can delegate more complex tasks to agents without sacrificing quality. The key is treating verification as a system that needs maintenance, tuning, and protection from the very agents it evaluates.

This approach also reduces the cognitive load on human reviewers. Instead of hunting for bugs, reviewers focus on high-level architecture and handling edge cases that automation misses. The team’s method of using "observe mode" for new gates allows them to tune sensitivity without blocking development, ensuring that false positives do not disrupt workflow. This balance between automation and human oversight is critical for sustainable AI-assisted development.

What you can do

  • Create a verify skill in your repository that defines how to boot and test your application end-to-end.
  • Prioritize writing E2E tests that cover critical user journeys rather than relying solely on unit tests.
  • Implement property tests for core logic to catch edge cases using randomized input generation.
  • Store verification rules and review guidelines in files like REVIEW.md on protected branches.
  • Configure agents to run local tests and linters before opening pull requests to save CI resources.
  • Regularly review reverted changes and failed tests to update your verification rules and prevent recurrence.
  • #ai agents
  • #software testing
  • #claude code
  • #verification
  • #engineering practices

Tools from the Bytechap store

Keep reading

All stories