Compaction without losing the thread
Compaction without losing the thread is the practice of replacing a long agent interaction history with a smaller working state that still preserves intent, constraints, decisions, evidence, tool results, failures, and pending work, so the next model call can continue the task correctly rather than merely recall a vague summary.
Long-running agent tasks accumulate more context than a model can use well. Old messages contain useful facts, obsolete attempts, noisy tool output, and conversational filler mixed together. If everything is kept, relevant details can be buried or pushed out. If history is naively trimmed, the agent may repeat failed work, violate a user constraint, forget why a decision was made, or lose track of unresolved questions.
Compaction works by turning the transcript into an explicit state object or structured brief. Instead of summarising every turn evenly, it separates durable facts, current goal, accepted decisions, rejected approaches, open questions, tool results, file names, errors, and next actions. The compacted state is then inserted into future prompts in place of much of the raw history, sometimes with links or stored artefacts retained elsewhere.
The trade-off is that compaction is lossy, and the loss is only safe if it matches the task. A polished narrative summary can be worse than a messy but precise handoff if it omits exact commands, API responses, or commitments. More structure costs design effort and tokens, while too little structure makes the model infer continuity. The right boundary depends on the agent’s job, tools, risk, and how often state changes.
Engineers meet this in agent frameworks, chat memory layers, trace debugging, and production support for long tasks. It appears as summarisation middleware, rolling memory, checkpoint prompts, scratchpad pruning, or custom state stores. When an agent goes off track, inspect whether it lacked necessary context, carried too much irrelevant context, or received a compacted summary that quietly changed the meaning of the work.
Common questions
- Is this just conversation summarisation?
- Not quite. Conversation summarisation usually tries to describe what was said. Compaction for agents tries to preserve what will affect future action. That includes constraints, decisions, failed attempts, exact artefact names, tool outputs, pending tasks, and reasons. A readable summary is useful only if it keeps the operational state intact.
- What should be kept in compacted context?
- Keep the user’s goal, non-negotiable constraints, current plan, important decisions and their reasons, facts learned from tools, errors already encountered, rejected options, pending questions, and promised next steps. Keep exact identifiers when they matter, such as filenames, endpoint names, configuration keys, or returned values. Drop greetings, duplicates, stale exploration, and tool noise with no future relevance.
- How do I know whether compaction caused an agent failure?
- Compare the raw trace with the compacted state the model actually saw. If the missing detail explains the bad action, compaction dropped too much. If the detail was present but buried among irrelevant material, the problem may be context noise. If the compacted state says something subtly different from the trace, the summary broke continuity.