Español

Add One Line to Every AI Research Prompt

Your AI research is often just confirmation bias with citations.

Ask a large language model a broad question — "should we adopt X?", "is Y a good strategy?" — and it tends to hand back a well-sourced case for whatever your question already implies you want. The links look authoritative. The reasoning reads clean. But you asked for a verdict you were leaning toward, and the model, trained to be agreeable and to complete the pattern you set, gives you the upside and quietly drops the rest. That's not research. It's research theater.

Why the model agrees with you

Two forces push a model toward your prior. First, framing leaks intent: "why is X the right choice?" primes a different answer than "when does X fail?" The model completes the frame you handed it. Second, alignment training rewards helpfulness and agreeableness, so models lean toward confirming rather than challenging. The result is a subtle sycophancy: it surfaces supporting evidence, downplays counter-evidence, and rarely volunteers the strongest case against your position unless you demand it.

The danger is that citations make the bias feel rigorous. You walk away more confident and less informed.

The fix: one line, every time

Append a single instruction to every research prompt:

Include the strongest contrarian evidence and the main downside cases.
Present the best argument against this position, not just for it.

That one line changes the retrieval and generation target. Now the model has to go find and state the risks, the failure modes, and the credible dissent — right next to the upside. You're no longer reading a brief written to persuade you; you're reading both sides and deciding on the whole picture.

A slightly longer version that works well for higher-stakes questions:

For this question:
1. Give the strongest case FOR.
2. Give the strongest case AGAINST, steelmanned.
3. List the conditions under which the AGAINST case wins.
4. Flag where the evidence is thin or contested.

Power moves

• Make it a system prompt or saved instruction, not something you retype. If every research conversation starts with "surface counterarguments and downside cases," you never forget on the day it matters most.
• Ask for the disconfirming test. "What evidence would change your recommendation?" forces the model to name a falsifiable condition instead of hedging.
• Separate the roles. Run one pass as an advocate and a second, fresh pass as a skeptic, then compare. Splitting the two jobs beats asking one answer to be balanced.
• Watch for fake balance. A token "however, some critics say…" is not a downside analysis. Push until the against-case is as concrete and sourced as the for-case.

Why it works

You are not trying to make the model pessimistic. You are correcting for a structural bias by changing the target it optimizes toward. When "good answer" is defined as decision-useful rather than agreeable, the model does the harder, more valuable work: showing you the risks and counterarguments alongside the upside, so you own the whole trade-off instead of a curated slice of it.

Resources

• Anthropic — Sycophancy in language models
• OpenAI — Prompt engineering guide
• Confirmation bias — Stanford Encyclopedia of Philosophy
• Steelmanning — LessWrong concept reference