The 5-Second AI Hallucination Detector

AI-generated code has a specific failure mode: it can look completely correct — clean naming, sensible structure, plausible comments — while calling a function that has never existed in any version of the library. You won't catch it by reading. Your type checker will catch it in seconds.

Why AI code lies convincingly

A language model generates code the same way it generates prose: by producing what's statistically plausible. If a library should have a client.batchUpdate() method — because similar libraries do, because the naming pattern fits — the model will happily call it whether or not it exists. It blends APIs across library versions, invents keyword arguments that were never accepted, and imports symbols from the wrong module.

The dangerous part is that hallucinated code passes the eyeball test. It's formatted like real code, named like real code, and surrounded by code that is real. Human review is tuned to catch bad logic, not nonexistent symbols — nobody memorizes a dependency's full API surface. So the fake call sails through review and fails at runtime, or worse, in production.

The 5-second check

A type checker resolves every symbol against what actually exists — your code, your installed dependencies, their type definitions. A hallucinated method is, to a type checker, just an unresolved reference. It gets flagged instantly, with a file and line number.

# TypeScript — type-check only, build nothing
npx tsc --noEmit

# Python — static check, nothing executes
npx pyright

The key is that this is a pure check. No build, no bundling, no test run, nothing executed. tsc --noEmit type-checks the whole project and writes zero output files; pyright statically analyzes Python the same way. On a typical project either finishes in seconds — which is why it belongs immediately after every AI generation, not just at CI time.

Make it the habit

The loop is: AI writes → types verify → you review. The type checker goes in the middle deliberately — it's cheap and mechanical, so let it burn down the mechanical errors before your expensive human attention starts. Your review time then goes where it's actually needed: logic, edge cases, design.

To make the habit stick, take it out of your hands: add the check as a pre-commit hook. Then no code — AI-written or otherwise — reaches a commit with unresolved symbols in it, regardless of whether anyone remembered to run the check.

Power tricks

• Run it in watch mode (tsc --noEmit --watch, pyright --watch) in a side terminal while the AI works. Hallucinations surface the moment the file is saved.
• Fail CI on type errors. The pre-commit hook can be skipped; CI can't. Make the type check a required, blocking step.
• Turn on strict mode. TypeScript's strict: true and pyright's strict setting catch a whole extra class of AI mistakes — implicit any leaks, unhandled None, mismatched optionals — that loose configs let through.

Resources

• TypeScript — the noEmit compiler option
• Pyright — static type checker for Python