Make Your AI Say UNVERIFIED
The most expensive thing your AI does is sound confident. A hallucinated method signature — client.stream(on_delta=...) for a callback that never existed — reads exactly like a real one. There's no red squiggle, no warning. You paste it, wire it up, and lose an hour tracing an AttributeError back to a function the model invented.
Why confidence is the bug
OpenAI's 2025 analysis puts a number on it: their audit of major benchmarks found 9 of 10 used binary grading that rewarded a confident guess and penalized "I don't know." A model trained and scored under those incentives learns that guessing pays. So when it doesn't know the exact flag name, it doesn't hedge — it emits the most plausible token sequence, which for API code is a signature that looks perfectly idiomatic and is completely fake.
Plausible is the whole problem. Prose hallucinations you can smell. A framework-specific parameter you can't, until it throws at runtime.
The fix: cite or say so
Give the model an explicit third option besides "answer" and "guess": abstain, and mark it. Two rules in your system prompt do most of the work.
- Every framework-specific pattern must cite an official doc. Method names, config keys, CLI flags, decorators — if it can't point to a source, it can't assert the line as fact.
- When it can't verify, it writes
UNVERIFIED. Not silence, not a hedge buried in prose — a literal token you can grep for before the code reaches prod.
Anthropic's guidance is the same instinct, stated three ways:
| Technique | What you tell the model |
|---|---|
| Allow uncertainty | "If you're unsure, say 'I don't have enough information to confidently assess this.'" |
| Verify with citations | Cite a source for each claim; if you can't find one, retract the claim |
| Restrict knowledge | Use only the provided docs, not general knowledge |
That middle row is the reel's rule almost verbatim: an unsupported claim gets removed, not smoothed over.
A prompt you can paste
For any framework-specific API (method names, parameters, config keys,
CLI flags), cite the official doc URL you're relying on. If you cannot
verify a pattern against official docs, do NOT guess — write the line
and tag it: # UNVERIFIED: confirm against docs before prod.
Prefer an honest "I don't know" over a plausible invention.Now the failure mode is loud. Instead of a fake signature hiding in 40 lines, you get a # UNVERIFIED marker and a doc link to check. The hour you would have spent debugging becomes a ten-second grep.
For power users
- Extract quotes first. For long docs (>20k tokens), Anthropic recommends asking the model to pull word-for-word quotes before it answers, then reason only from those quotes. Grounding beats recall.
- Post-hoc retraction pass. After a draft, have the model re-read each claim and find a supporting quote; anything it can't support gets deleted and marked with empty
[]. Cheap second pass, catches confident leftovers. - Restrict to provided docs. Explicitly forbid general knowledge for the task — "use only the pasted docs." Turns off the exact recall that produces plausible fakes.
- temperature=0 for factual code. Lower sampling temperature narrows the model toward its highest-confidence tokens, which for grounded tasks means fewer creative inventions.
- Grep the diff for
UNVERIFIEDin review or CI. A marker nobody checks is just a comment.
The point isn't to make the model smarter. It's to make its ignorance visible. Honesty about what it couldn't verify beats false confidence every single time.
Resources
- Reduce hallucinations — Claude Docs — allow "I don't know", verify with citations, retract unsupported claims
- Why language models hallucinate — OpenAI — the incentive to guess
- Why language models hallucinate (paper, PDF) — Kalai, Nachum, Vempala, Zhang
- Evaluating LLMs for accuracy incentivizes hallucinations — Nature — the benchmark-grading finding
<div class="cta">
Building an AI feature? Yeda AI designs, audits, and ships production LLM systems that fail loud, not silent.
</div>