Context rot — why long inputs are used worse than short ones
In one paragraph
Everything inside the context window is available to the model, but it is not used evenly. Published measurements show accuracy falling when the relevant part sits in the middle of a long input, and falling again simply because the input is longer — even when the model can retrieve every relevant fact perfectly. Vendors now call it context rot.
the same task, in a longer and longer chat
the shape, not the data — the measured drops, with their study, are in the text below
Published accuracy by the position of the answer inside the input. With 30 documents, GPT-3.5-Turbo answered 73.4% when the answer came first, 50.5% when it sat tenth, and 63.7% when it came last — the U shape that gave the effect its name.
Two different effects with the same feel
Position. Lost in the Middle held everything constant except where the answer sat inside the input. The finding:
performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models
The chart above is those published numbers redrawn — with thirty documents, the same model scored 73.4% when the answer was first and 50.5% when it was tenth.
Length. A later result removed position from the equation entirely. In Context Length Alone Hurts LLM Performance Despite Perfect Retrieval, the authors report that “even when models can perfectly retrieve all relevant information, their performance still degrades substantially (13.9%–85%) as input length increases but remains well within their claimed context lengths”. The drop survived replacing the irrelevant text with whitespace and survived masking it altogether.
Independent testing joins the two. Chroma evaluated eighteen models, including current frontier ones, and found that “models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows”, in ways that are “surprising and non-uniform”.
Why the vendors stopped arguing
Anthropic’s documentation now states it as a design constraint rather than a caveat — “more context isn’t automatically better”, because:
as token count grows, accuracy and recall degrade, a phenomenon known as context rot
Its conclusion for anyone building on top is the same one that applies to you at a keyboard — this “makes curating what’s in context just as important as how much space is available”.
That is the whole practical content of the idea. A window is a capacity, not a promise. Filling it is possible, cheap and usually a mistake.
Technical detail: one documented mitigation
The EMNLP 2025 authors propose a fix that needs no tooling: prompt the model to recite the retrieved evidence before it attempts the problem, which converts a long-context task into a short-context one. On the RULER benchmark they report a consistent improvement of GPT-4o of up to 4% over an already strong baseline.
In a chat window that is a single sentence — ask it to quote the passages it is relying on before answering. It also fails loudly when there is nothing to quote, which is worth as much as the accuracy.
Common questions
- 1 Are the numbers in the chart current?
- No, and we label them rather than dress them up. They come from models measured in 2023 on a multi-document question-answering task. The shape is the published finding; the values are not a measurement of anything you can use today. Later independent work on eighteen newer models reproduced the direction, not the numbers.
- 2 Does a larger context window solve it?
- It raises what fits. The EMNLP 2025 result was measured on inputs well inside the models' claimed context lengths, so the degradation was already present where there was plenty of room. Anthropic's own documentation says more context is not automatically better.
- 3 Is this the same as the chat dropping old messages?
- No. Dropping happens when the transcript outgrows the window and is a hard, mechanical cut. This effect appears while everything still fits and nothing has been removed.