Español

The Bug in an AI PR Lives in the Callers

The bug in an AI-generated PR usually isn't in the diff. It's in the callers. The diff looks locally reasonable and the tests pass, so you approve it — and then a changed signature or a narrowed type breaks a consumer three files away that the diff never put in front of you.

Why AI PRs fail this way

An agent optimizes for the task it was handed: make this function correct, make these tests green. It edits inside a window and stops. What it rarely does on its own is enumerate every place that depends on what it just changed. So the change is right where you're looking and wrong where you're not.

The data backs this up. In CodeRabbit's 2025 analysis of hundreds of thousands of pull requests, AI-assisted PRs surfaced 10.83 findings on average versus 6.45 for human-authored ones — about 1.7x more issues — with roughly 75% more logic-and-correctness findings and algorithm/business-logic errors appearing more than twice as often. "Green but globally broken" is not an edge case; it's the modal AI-PR failure.

Scope your review to the blast radius

The fix is a discipline, not a tool. For every symbol the PR changes — a function, a type, a constant, an exported name — do three things:

1. List what changed. Read the diff and write down each renamed, retyped, or re-signatured symbol.
2. Find its callers. Search the whole repo for who imports or calls each one.
3. Confirm they still hold. Open each call site and check that the new signature, type, or contract still satisfies it.

Find the callers fast

git grep is the quickest way to enumerate a symbol's references across the tree. It searches the working tree, the index, or any commit:

# Every reference to the changed symbol
git grep -n "processOrder"

# Whole-word matches only, with the enclosing function for context
git grep -nW -w "processOrder"

-n prefixes line numbers, -w matches only at word boundaries (so processOrder doesn't also hit processOrders), and -W (--function-context) prints the whole function each match sits in, so you read the call in context instead of one naked line.

Power moves

• Let the type checker walk the graph for you. In a typed language, a changed signature that breaks a caller is a compile error, not a test you forgot to write. Run the full-project type check (tsc --noEmit, mypy, cargo check) before you trust green unit tests — it visits every caller, which is exactly the set the diff hides.
• Ask the agent to enumerate its own blast radius. Before you review, prompt it: "List every call site of each symbol you changed and confirm each still type-checks." You're turning the silent step into an artifact you can audit.
• Prefer full-repo review tooling for cross-file changes. Line-diff review tools see only the hunk; codebase-indexing reviewers reason across files and catch signature mismatches that span modules. Match the tool to the change: a self-contained fix and a wide refactor need different scrutiny.

Resources

• git grep — official documentation — the -n, -w, and -W/--function-context flags.
• CodeRabbit's AI-assisted PR report (Help Net Security) — the 10.83-vs-6.45 findings and 75%-more-logic-errors numbers.
• How I Review AI-Generated PRs: a step-by-step checklist — jump-to-definition and cross-file signature checks.
• Reviewing AI-Generated Code: different PRs need different patterns — scaling review scrutiny to the change's reach.