Yeda AI Tips · #094

Español

Have a Separate Subagent Write the Reproduction Test

Your AI grades its own homework. When the same model, in the same conversation, writes both the bug fix and the test that "proves" it, the same assumptions carry through: whatever the model misunderstood about the bug shapes the fix and the test, and they agree — wrongly. The fix is one prompt away: split the job.

Why one agent's test can't catch its own fix

An agent stops when the work looks done, and "looks done" is a weak signal when the checker shares the worker's reasoning. Anthropic's own Claude Code guidance is explicit about the mechanism: a verification pass in a fresh context works because "the agent doing the work isn't the one grading it," and a reviewer in a fresh context "sees only the diff and the criteria you give it, not the reasoning that produced the change." The same doc notes that a fresh context improves code review because the model won't be biased toward code it just wrote.

Concretely, a self-graded bug fix fails in two boring, common ways:

  1. The test encodes the misdiagnosis. If the agent thinks the bug is an off-by-one when it's actually a timezone issue, it writes an off-by-one test — which the wrong fix happily passes.
  2. The test is written to pass. An agent that already knows the fix tends to assert exactly what the fix produces, not what correct behavior requires.

The workflow: prove the bug first

StepWhoWhat
1Fresh subagentGets only the bug report + repro steps. Writes a failing test. Never sees the fix.
2You / the harnessRun the test. It must fail for the reported reason — that failure is the reproduction.
3Main agentImplements the fix, iterates until the subagent's test passes.
4Optional reviewerA second fresh subagent reviews the diff against the test.

In Claude Code, subagents run in their own context window, so isolation is the default — the test-writer literally cannot peek at the fixer's reasoning. The prompt is one line:

Use a subagent to write a failing test that reproduces this bug:
users report login fails after session timeout. Give it only this
report — not the fix branch. Run it and show me the failing assertion.

Then, in the main session: "make this test pass without modifying the test."

Verify the failure, not just the test

A test that fails for the wrong reason (import error, missing fixture) proves nothing. Make the subagent show evidence: the exact command it ran and the failing assertion. Only when the failure matches the reported symptom do you have a reproduction — then hand it to the fixer.

Power-user moves

Resources

Building an AI feature? Yeda AI designs, audits, and ships production LLM systems.

Talk to us · Read the blog