Context rot is the drop in a language model's accuracy and recall as the number of tokens in its context window grows. A July 2025 Chroma report that tested 18 models found performance degrading with input length even on simple tasks, so what an agent sees matters more than how much fits.

Context engineering and AI cost

What is context rot?

Alexandru Lupascu, CTO & Co-founder · Updated 23 October 2026

Context rot is the drop in a language model's accuracy and recall as the number of tokens in its context window grows. Anthropic's context window documentation uses the term in that sense: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot." A July 2025 Chroma technical report titled "Context Rot: How Increasing Input Tokens Impacts LLM Performance" (Hong, Troynikov and Huber) tested 18 models and found that their performance "grows increasingly unreliable as input length grows." For AI agents, context rot means a long session can get less accurate even when the right facts are still in the window.

What does the research on context rot show?

The clearest evidence comes from tests that keep the task fixed and only add tokens. In the Context Rot report, models answered questions about a long chat history (LongMemEval, 306 prompts) in two forms: a focused version of about 300 tokens and a full version of about 113,000 tokens containing the same answer. "Across all models," the authors report "significantly higher performance on focused prompts." Even one distractor, a sentence that looks relevant but does not answer the question, lowered performance, and four distractors lowered it further. Position matters too. "Lost in the Middle" (Liu et al., TACL 2023) found accuracy highest when the relevant passage sat at the start or end of the input; in the worst case, with 20 or 30 documents, GPT-3.5-Turbo scored below its 56.1% accuracy with no documents at all.

Why does context rot happen?

Context rot follows from how transformers spread attention. Anthropic's engineering team explains that every token attends to every other token, which creates n² pairwise relationships for n tokens, so the model's "attention budget" is stretched as the context grows. Models also train mostly on shorter sequences, so they have "less experience with" dependencies that span the whole context. The result, in Anthropic's words, is "a performance gradient rather than a hard cliff": models stay capable at long lengths but lose precision in recall and long-range reasoning. The Context Rot report adds that the kind of content matters as well as the amount: across all 18 models, performance was better on shuffled text than on logically structured text of the same length.

How do you reduce context rot in an AI agent?

You reduce context rot by sending the model less and better context on each turn:

Keep only what the current step needs. Anthropic's rule is "the smallest possible set of high-signal tokens."
Clear old tool results once the agent has used them.
Compact long sessions, and check what the summary drops (see what Claude Code forgets when it compacts).
Remove distractors: outdated versions of a fact, near-duplicates and superseded decisions.
Put the instruction and the key facts at the start or end of the input, not in the middle.
Test on your own tasks: run the same task with a full and a trimmed context and compare the results.

Where does elephant.broker fit?

elephant.broker works on the input side of context rot: on every turn the runtime ranks and prunes the retrieved context, keeps what is current and verified, drops stale or duplicate facts, and places the working set where the model pays attention, with your model and harness unchanged. elephant.broker reports on its website 40-80% less context per task, with "only verified, relevant knowledge" reaching the window. That is elephant.broker's own reported result, not an independent or head-to-head test. The trade-off is an extra retrieve and rerank step per turn. The general method is covered in what context engineering is.

Sources

Anthropic: Context windows (documentation), Effective context engineering for AI agents, published 29 September 2025
Chroma (Hong, Troynikov and Huber): Context Rot: How Increasing Input Tokens Impacts LLM Performance, technical report, 14 July 2025
Liu et al.: Lost in the Middle: How Language Models Use Long Contexts, TACL 2023
elephant.broker: overview, how a turn flows

Product details come from each vendor's public documentation as viewed on 2026-10-23 and may have changed. Product names are trademarks of their respective owners; elephant.broker is not affiliated with them.