How do I reduce hallucinations in AI agents?

You reduce hallucinations in AI agents by changing what the model sees and what it is rewarded for: ground each answer in sources it can quote, give it explicit permission to say "I don't know", keep stale and conflicting facts out of its context, and verify claims before the agent acts on them. Anthropic's "Reduce hallucinations" guide says that allowing uncertainty "can drastically reduce false information", and also that these techniques "don't eliminate them entirely." A September 2025 paper by researchers at OpenAI and Georgia Tech explains the root cause: training and evaluation reward guessing.

Why do AI agents hallucinate?

Language models hallucinate partly because guessing scores better than abstaining. Kalai, Nachum, Vempala and Zhang ("Why Language Models Hallucinate", arXiv 2509.04664) argue that "the training and evaluation procedures reward guessing over acknowledging uncertainty": under right-or-wrong grading, "I don't know" earns nothing, so a confident guess is the better bet. Retrieval is not a full fix either, because that grading "still rewards guessing whenever search fails to yield a confident answer." Agents add a second cause: their context. The Context Rot report (Hong, Troynikov and Huber, July 2025) found that even a single distractor, a statement that looks relevant but does not answer the question, lowered performance, and distractors showed up in the models' hallucinated answers. In an agent, outdated facts and superseded decisions act as distractors.

How do I reduce hallucinations step by step?

Apply these in order, starting with the prompt and ending with checks on high-risk actions:

1. Allow uncertainty. Add a line such as: "If the sources do not contain the answer, say 'I don't have enough information' and stop."
2. Quote before answering. For documents over 20k tokens, Anthropic recommends extracting word-for-word quotes first, then answering only from them.
3. Cite and retract. Ask for a supporting quote per claim, and remove any claim without one.
4. Restrict the sources. Tell the agent to use only the provided documents and tool results, not general knowledge.
5. Keep the context current. Drop superseded facts, and when two sources disagree, show both with their dates (see conflicting information in agent memory).
6. Check high-risk outputs. Run the same prompt several times and treat disagreement as a warning, as in Anthropic's best-of-N verification, and require human approval before irreversible actions.

How do I measure whether hallucinations went down?

Measure wrong answers and abstentions separately, because a single accuracy score rewards guessing. The OpenAI and Georgia Tech authors propose stating a confidence target in the instructions: answer only if you are more than t confident, since "mistakes are penalized t/(1-t) points" while a correct answer earns 1 point and "I don't know" earns 0. At t = 0.9 a wrong answer costs 9 points, so answering pays only above 90% confidence. Build a test set from your agent's real tasks, include questions whose answer is not in the sources, and track three numbers per release: correct, wrong and abstained. Re-run it whenever you change the prompt, the retrieval or the model.

Where does elephant.broker fit?

elephant.broker works on the context side of hallucinations: on every turn the runtime keeps current, verified context, drops stale or duplicate facts, and when two sources conflict, places both and weighs them by proof instead of silently picking one. elephant.broker reports on its website +36% better accuracy, with stale-state errors falling from 47% to 11%. That is elephant.broker's own reported result, not an independent or head-to-head test, and no context change removes hallucinations entirely. The runtime adds a retrieve and rerank step per turn and records every read, placement and write-back in an audit log, so you can trace a wrong answer to what the model saw. Related: context rot and context engineering.

Sources

• Anthropic: Reduce hallucinations (documentation)
• Kalai, Nachum, Vempala and Zhang: Why Language Models Hallucinate, arXiv, 4 September 2025
• Chroma (Hong, Troynikov and Huber): Context Rot: How Increasing Input Tokens Impacts LLM Performance, technical report, 14 July 2025
• elephant.broker: overview, features

Product details come from each vendor's public documentation as viewed on 2026-10-28 and may have changed. Product names are trademarks of their respective owners; elephant.broker is not affiliated with them.