Conversation compaction is the process of shortening an AI chat history while preserving the information needed for the next step. In agentic systems, it reduces token use and keeps context manageable by summarizing prior messages, decisions, and constraints, so the model can continue operating without losing critical state.
What conversation compaction does in an agentic system
Conversation compaction is not just summarisation, it is state management. The goal is to preserve the decisions, constraints, and unresolved tasks that still matter while removing low-value history that would otherwise consume context and slow the model down.
That makes it especially useful in long-running agentic workflows where the conversation is effectively part of the system state. Good compaction keeps the next step grounded in what has already been decided, rather than forcing the model to re-read every prior message.
Why compaction matters for reliability and token efficiency
Without compaction, chat history grows until the model spends more of its context window on stale material. That can increase cost, reduce responsiveness, and make important instructions easier to bury under irrelevant detail.
Compaction is therefore a trade-off between completeness and working memory. If it is too aggressive, it can drop commitments, constraints, or assumptions that later steps depend on; if it is too conservative, it fails to solve the token-pressure problem it was meant to address.
What should be preserved when history is shortened
The most important content to carry forward is the information that changes future behaviour: user intent, policy constraints, task status, selected options, dependencies, and any facts the agent must not forget. Repeated wording, earlier brainstorming, and dead-end branches are usually better candidates for removal.
Compaction works best when the summary is selective and structured, not merely shorter. A useful compacted record should still tell the model what it is doing, what has already been ruled out, and what must remain true in the next turn.
Common failure modes in compaction
Conversation compaction can fail when the summary captures topics but misses commitments. A reduced history may look complete while quietly losing a constraint, a user preference, or a dependency that later changes the correct answer.
It can also distort the conversation if the summariser over-interprets the dialogue. In agentic settings, that is particularly risky because one bad summary can propagate through multiple future actions, making the system act on an incomplete or slightly wrong version of its own history.
Risk and Threat Considerations
Compaction creates a control point where important context can be lost, mis-stated, or selectively omitted. In agentic systems, that can turn a harmless shortening step into a persistence, safety, or integrity problem if later actions rely on the compacted state as if it were authoritative.
Failure mechanism: An attacker, faulty summariser, or poorly designed compaction policy can cause important instructions, constraints, or prior decisions to disappear from the retained context, allowing the system to behave as if they never existed.
Impact: The agent may repeat already-resolved work, ignore user boundaries, carry forward the wrong task state, or make unsafe tool and action choices because the memory it trusts no longer matches the original conversation.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI06 — Memory & Context Poisoning | Conversation compaction directly manages retained agent context and state. |
| Recommendation — Preserve critical state and verify compaction outputs so later agent steps do not rely on poisoned or incomplete context. | ||
| NIST SP 800-53 Rev 5 | AU-6 — Audit Review, Analysis, and Reporting | Compaction benefits from review of what state was retained or lost across turns. |
| IA-5 — Authenticator Management | Agent conversations often carry credentials, tokens, or secrets that must not be retained unnecessarily. | |
| AC-6 — Least Privilege | Compacted context should preserve only the authority and permissions needed for the next step. | |
| Recommendation — Review compacted conversation records to detect when key decisions or constraints were dropped. Control retention and rotation of any secret material that appears in conversation state. Minimise retained authority cues so the agent acts with only the access needed for the current task. | ||
Practitioner Guidance
What to preserve: Keep compaction focused on durable state, not on generic summary length. The retained record should explicitly carry forward goals, open decisions, constraints, and any assumptions that would change the next action if removed.
What to watch for: Treat compaction as a governed transformation, not an automatic cleanup step. The strongest implementations make it easy to inspect what was dropped, what was retained, and whether the reduced context still supports the next task without ambiguity.
Related resources from NHI Mgmt Group
- What is the difference between prefix caching and conversation compaction for AI agents?
- Why does MCP change the IAM conversation for agents?
- What changes when an AI chat system can switch between different models mid-conversation?
- What breaks when conversation state is spread across local storage, proxies, and external model calls?