Español

Your Agent Can't Test What You Never Gave It

Your agent implements a feature, writes a unit test, declares it done, and never once opens the thing it built. It reads like laziness, or a model that doesn't care. It's usually neither. It verified the one layer it could actually reach, and stopped at the edge of what you gave it.

Reach, not diligence

Think about what end-to-end verification physically requires for a typical change. A browser to click through the flow. A command-line tool for whatever service is involved. A test account to log in as. Credentials that are safe to use. A way to read the logs afterwards.

Now ask how many of those your agent has. For most setups the honest answer is the first zero or one. Unit tests are frequently the only verification available to it, so unit tests are what you get — not because it judged them sufficient, but because everything past that was a wall.

This reframes a complaint into a task. "My agent doesn't test properly" is a statement about your environment more often than about the model. And the environment is fixable, permanently, in an afternoon.

Ask it, rather than guessing

You don't have to work out the gap list yourself, and guessing tends to produce the obvious items while missing the ones that actually block it. Describe the kind of change you make most often and ask directly what it would need in order to verify that end to end, unaided.

"When I ask you to change the checkout flow, what would you need
 from me to verify it end to end yourself, without me checking?
 List the tools, accounts, and credentials, and say what each unblocks."

The answers are usually concrete and cheap: a CLI you never installed, a throwaway account, a scoped API key, a way to reach a staging copy. Occasionally something genuinely awkward surfaces — a hardware dependency, a service with no test mode — and knowing that is also valuable, because you can stop expecting verification there and put a human check in deliberately.

Build it once; every run inherits it

This is the part that makes it worth an afternoon. A verification environment is not per-task work — it's a durable asset. Install the tools, provision the throwaway account, mint the scoped keys, and every future agent run on that codebase starts with the ability to check itself.

Two things worth doing while you're there:

• Scope the credentials to the least destructive mode the task needs. Read-only where reading suffices; a disposable copy rather than anything real. An agent with a verification environment is an agent with more reach, and reach deserves narrower permissions.
• Write down how to use each one. A tool the agent can't discover is a tool it won't use. Whether that lives in a rules file or a skill, the invocation needs to be findable.

Then make the check mandatory

Capability alone doesn't change behavior — an agent that can verify still needs a reason to. Giving verification targets is what lets it catch problems before you have to ask for fixes, so state the check in the same prompt, or escalate to something enforced when it matters.

The companion article on verification layers covers the escalation path: the check in the prompt, a goal re-evaluated each turn, a hook that blocks the turn until your script passes, and an adversarial reviewer. All four assume the agent can actually run something. That assumption is what this article is about.

The takeaway

Your agent isn't a bad tester. It's a tester you locked out of the building. Ask what it needs to verify your most common change end to end, install exactly that list once, then require the check — and the shallow-testing complaint stops being about the model.