05.05 · Concept
Context rot
Recognise when more context is making answers worse, and prove it.
Context rot is degradation caused by stuffing an LLM prompt with material that technically fits but makes the right evidence harder to use. Irrelevant, stale, duplicated, or conflicting text can pull answers towards the wrong source. The reliable test is an ablation: hold the question and model fixed, then compare cleaner and noisier context variants.
What this lesson answers
- how does too much context hurt LLM answers
- how to prove context rot with an evaluation
- when is long context worse than retrieval
Notes
Context rot is what happens when an LLM gets so much surrounding material that useful evidence is diluted by irrelevant, stale, duplicated, or conflicting text. The model can still technically “fit” the input inside its context window, but fitting is not the same as using it well. More tokens increase the chance that the answer is influenced by the wrong passage, an old instruction, a near-duplicate, or a plausible but irrelevant detail.
A useful mental model is debugging with a giant log file. If the log contains only the ten lines around the failure, the cause may be obvious.
Common questions
- What is context rot in an LLM application?
- Context rot is what happens when extra prompt material makes an answer worse instead of better. The model may receive all the text successfully, but irrelevant, obsolete, duplicated, or contradictory passages compete with the useful evidence. The result is often a plausible answer grounded in the wrong part of the supplied context.
- Does a larger context window prevent context rot?
- No. A larger context window only means more input can be supplied. It does not guarantee the model will rank the right passages, ignore stale instructions, or resolve contradictions correctly. Long context helps when the added material is necessary and well organised. It hurts when it adds noise around the actual evidence.
- How can I prove that more context is making answers worse?
- Use an evaluation set with known expected answers. Keep the model and question fixed, then compare runs with minimal relevant context, added irrelevant material, stale conflicting material, and the full production prompt. Track correctness, refusals, citation quality, or a task-specific pass condition. A consistent drop under noisier conditions demonstrates context rot.
Short definition: what is Context rot?