Context Rot in LLMs: Why LLMs Get Worse With Longer Inputs

10 min read

This analysis draws on published research and reconstructed multi-session workflows. Vendor and single-study figures are identified in the text.

TL;DR Tap to expand the short version

Context rot is a measured, in-session problem. The main premise is that as the input you feed an LLM model grows, its output gets less reliable, even on simple tasks. Chroma tested 18 frontier models in 2025 and every one degraded as input length climbed.
Context loss is the operational knowledge that disappears when a session ends, when a model is swapped, when a tool changes, or when one person hands work to another.
Both compound into context debt. It’s the quality drop that show up once your team moves past one-off queries into multi-day work.
The usual advice, write a summary, use the memory feature, start a fresh chat, is a partial patch, not a fix, because it loses the decisions and rejected paths that made the work good.
You can measure your own context debt in two weeks with four numbers, and reduce it with patterns that travel across tools instead of locking you into one vendor’s memory feature.

A campaign plan, contract review or financial model may span several AI sessions. The model learns the constraints, previous decisions and rejected options. Then the session ends, a usage cap appears or the team changes tools.

The next model receives the files and perhaps a summary. It does not receive the reasoning that shaped the work. The team must rebuild that understanding, usually more slowly and less accurately than the first time.

Two failures cause this problem. Context rot happens inside a growing session. Context loss happens when work crosses into a new session, tool or person.

Defining context rot

Context rot is the decline in model reliability as input grows, often well before the context window is full. Chroma’s 2025 study tested 18 frontier models on simple tasks at increasing lengths; every model became less reliable as more text was added.

Stanford-led research documented a related “lost in the middle” effect in 2023. models retrieve information more reliably at the beginning or end of an input than from the middle. Large advertised context windows therefore measure capacity, not consistent understanding.

Longer input can weaken a model before the advertised limit is reached. That measurable decline is context rot.

Two context failures

Context loss is operational. It occurs when a session ends, a usage limit interrupts work, a team changes tools or a colleague takes over. Without an external record, the next session starts cold.

Confusing the two leads to the wrong fix. If your problem is context rot, a shorter, cleaner prompt helps. If your problem is context loss, no amount of prompting inside a dead session brings the knowledge back, because the knowledge was never written down anywhere it could survive.

Files and final outputs move easily. The reasoning behind a decision does not. That is usually the most valuable part of the context.

Type of context	What it holds	Survives a switch?
Factual	Documents, data, prior outputs, links	Usually, if you re-attach it
Decision	Choices already made, options rejected, and the reasons for both	Rarely, this is the first to go
Constraint	Hard rules, brand voice, legal boundaries, risk appetite, style guides	Only if re-injected deliberately
Process	Current state of the work, next intended steps, open questions	Partly, and often out of date
Preference	How the operator likes to work, what has been tried, the model’s calibration to this project	Almost never

Teams tend to preserve the cheapest information and lose the part that made the work good. A new model can read the brief. It cannot recover an undocumented argument about why the obvious approach failed.

Forgetting costs

The cost is easy to feel and hard to see on a balance sheet, which is why it accumulates. Three bodies of evidence give it a shape.

The first is the cost of reorientation. When focused work is interrupted, getting back to full engagement takes time. The most cited figure, roughly 23 minutes to fully refocus, is associated with UC Irvine research, and note that that the precise “23 minutes and 15 seconds” number people repeat comes from her interviews rather than the printed paper.

The paper itself shows something arguably worse for quality. It found that interrupted workers finish tasks faster but under measurably higher stress and effort. A model switch is an interruption with extra steps, because you are recovering both your focus and the machine’s.

Prompts tuned for one model can lose more than 30% of their performance when moved to another. Multi-model use is already common. surveys indicate that many enterprises run several models and expect to depend on multiple providers.

These costs rarely appear on an invoice. Across a team using AI every day, they still become a steady tax on output.

Common failed fixes

Handoff notes and chat summaries help, but they usually capture outcomes rather than the reasons behind them. A rushed summary drops rejected options and hidden constraints first.

Projects, custom instructions and memory features improve continuity inside one provider. They do not travel with the work. Heavy dependence on them makes a later switch more expensive.

Pasting the full chat into a new session creates another problem. The long input buries the few decisions that need to survive and can trigger the same uneven attention seen in context rot.

Audit where context is lost

Most teams have never measured the cost of rebuilding context. For two weeks, track any workstream that spans several sessions and record what happens after each break, switch or handoff.

Minutes from reopening the work to your first real piece of new progress. This is your reorientation tax, and it is the single most important number.
A tally of choices you had to explain again because the model or the new session did not know them.
Count the outputs that broke a rule you had already established, such as a style guide, a legal boundary, or a format.
Roughly, what fraction of the session went to getting back up to speed versus making new progress.

Two weeks of this gives you four trend lines to act on. It also tells you which of the two problems you have. High reorientation time inside long single sessions points at context rot. High restated-decision and constraint-slip counts at the seams point at context loss. The fix is different for each , and that’s the whole reason this audit matters.

Preserving context

Move durable context out of conversations and into artefacts. Anthropic’s work on long-running agents uses the same principle. each session records its state so another model, tool or person can continue without reconstructing the work.

Keep a context object. One short living brief that holds the goal, the current state, and the hard constraints, stored outside any chat. It travels with the work and makes the conversation disposable, which is how it should be.

Keep a short context document outside the chat. Record the goal, current state, fixed constraints, important decisions and rejected options. Update it when the work changes.

The conversation can then be replaced without losing the project. Continuity lives in an artefact the team controls rather than in one model’s memory.

Always log decisions separately from chat. A short running record of what was chosen, what was rejected, and why. This is the single highest-value habit, because the rejected paths are what a summary always loses.
Try to maintain a constraint registry. The non-negotiable rules in one place, ready to be pasted back in deliberately at the start of any new session or tool. Do not trust the model to remember them across a seam.
Snapshot state at natural breakpoints. When you finish a phase, write the two or three sentences a stranger would need to continue. Do it at the breakpoint, not when you are already locked out and working from memory.
Run a survival check. After a switch, paste the context object into the new tool and ask it to restate the goal, the constraints, and the last decision. If it cannot, the object is incomplete, and you have found the gap.

The reason to favour patterns that live outside any one product is the same reason lock-in is a risk in the first place. If your continuity depends on a vendor’s memory feature, then treating context as managed context is what keeps a model switch from becoming a migration project.

Remaining limits

No mainstream tool provides professional-depth continuity across models, sessions and teams. Token limits, uneven attention and ecosystem lock-in all work against it.

Some context remains tacit. an experienced operator’s feel for phrasing and model behaviour is difficult to document. The practices here reduce that loss but cannot eliminate it.

Start this week

Choose one multi-session workstream. Maintain a context document outside the AI tool, run the Context Debt Audit for two weeks and use the survival check whenever a session or model changes. The process requires consistent documentation and review, not new software.

Context is the asset your team is building every time it works with a model. Right now, for most teams, it is an asset with no owner and no shelf. Giving it one is the cheapest quality improvement available.

FAQ

What is context rot in simple terms?

Context rot is the measured decline in a language model’s output quality as the amount of text you feed it grows. It appears even on simple tasks and well before the context window is full. A 2025 Chroma study tested 18 frontier models and found every one became less reliable as its input got longer.

Is context rot the same as losing context when I switch models?

No. Context rot happens inside a single session as input grows. Losing context when you switch models, tools, or sessions is a separate operations problem. the model retains nothing across those seams unless you carry it over deliberately. The fixes differ, which is why the distinction matters.

Why do my prompts stop working after switching to a different model?

Prompts are tuned to a specific model’s behaviour and do not transfer cleanly. Studies of prompt transfer have recorded meaningful accuracy drops, in some cases around a third, when a prompt is moved to a different model, though the size of the drop varies widely by model and task. Keeping your rules in a separate registry, ready to re-inject and re-test, turns that surprise into a quick check.

How can I measure how much context my team is losing?

Run the Context Debt Audit for two weeks on any workstream that spans multiple sessions. Each time you resume after a break, switch, or handoff, log four numbers. time to productive continuation, restated decisions, constraint slips, and the share of the session spent reorienting. The trend lines tell you whether your problem is in-session rot or cross-session loss.

Does using a tool’s built-in memory feature solve the problem?

It helps inside that one tool, but it does not travel. If your continuity depends on a single vendor’s memory feature, switching tools later becomes more expensive, and the decision reasoning that makes work good is often still lost. Storing context in artifacts you control, outside any chat, is what holds up across models and tools.

Spotted an error, have a correction, or want to pitch a story? Contact [email protected].

How AI Tool Switching Creates Context Loss

Table of Contents

Defining context rot

Two context failures

Forgetting costs

Common failed fixes

Audit where context is lost

Preserving context

Remaining limits

Start this week

FAQ

What is context rot in simple terms?

Is context rot the same as losing context when I switch models?

Why do my prompts stop working after switching to a different model?

How can I measure how much context my team is losing?

Does using a tool’s built-in memory feature solve the problem?

Like this:

How AI Tool Switching Creates Context Loss

Table of Contents

Defining context rot

Two context failures

Forgetting costs

Common failed fixes

Audit where context is lost

Preserving context

Remaining limits

Start this week

Related reading

FAQ

What is context rot in simple terms?

Is context rot the same as losing context when I switch models?

Why do my prompts stop working after switching to a different model?

How can I measure how much context my team is losing?

Does using a tool’s built-in memory feature solve the problem?

Share this:

Like this:

Discover more from Industry Contents