Context rot
The model getting less reliable as its context fills with stale, noisy, or competing material, even though the request still fits.
See it
What it is
Context rot is the informal name for answer quality degrading as a request fills with stale, redundant, contradictory, or weakly relevant material. The prompt can still fit inside the context window and still perform badly. The related 'lost in the middle' effect, named in a 2023 paper by Nelson Liu and colleagues, describes models missing useful information buried between a strong beginning and end.
Reach for the term when adding more chat history or retrieved documents makes results less faithful instead of more informed. Fix the input, not the wording around it: retrieve fewer chunks, rerank them, remove duplicates, summarize old turns, and start a fresh conversation when the old one no longer earns its space.
Gotcha: there is no universal token count where rot begins. It depends on the model, the task, where evidence sits, and how much the context disagrees with itself. A larger advertised context window only raises the hard ceiling; it does not make every token equally useful or turn a document dump into good retrieval.
Ask AI for it
Refactor this RAG pipeline to resist context rot. Retrieve 20 candidates, remove exact and near duplicates, apply Maximal Marginal Relevance to keep five diverse chunks, and order them by direct relevance to the question. Count the assembled request with tiktoken, preserve the system prompt and newest user turn, summarize old chat turns into a labeled block, and drop any chunk below the reranker threshold. Add an eval that compares answer accuracy and citation correctness at 5, 10, and 20 included chunks so extra context must prove that it helps.