Context capping

Expanded Definition

Context capping is a session-level control that restricts how much prior prompt, conversation, tool output, or retrieved state is reintroduced to a model on each turn. In agentic AI and NHI-adjacent systems, it helps bound token growth, reduce unnecessary reprocessing, and limit how much sensitive context can persist across interactions. It is related to memory management, but it is not the same as retention policy: retention decides whether data exists at all, while capping decides how much of it is fed back into active inference. Definitions vary across vendors because some systems cap by token count, some by message count, and others by semantic relevance or time window. NHI Management Group treats the control as part of a broader least-exposure posture, especially where tool calls, secrets, or privileged instructions may be replayed into an AI agent. The most common misapplication is treating context capping as a privacy control by itself, which occurs when teams cap token length but still preserve high-risk conversation state in retrievable memory.

For adjacent guidance on identity and AI risk framing, see NIST Cybersecurity Framework 2.0 and the NHI governance context in Ultimate Guide to NHIs.

Examples and Use Cases

Implementing context capping rigorously often introduces a usability tradeoff, requiring organisations to weigh conversation continuity against lower token cost and narrower data exposure.

• A customer-support agent keeps only the last few turns of the conversation, while older details are summarized or dropped before the next model call.
• A coding assistant limits retained build logs and prior instructions so it does not repeatedly ingest stale secrets or obsolete deployment steps.
• An AI agent that uses tools caps prior tool outputs to reduce accidental replay of credentials, tokens, or internal URLs into later reasoning steps.
• A regulated workflow combines context capping with explicit memory policy, ensuring only approved fragments persist beyond a single session.
• A retrieval-augmented system caps both chat history and retrieved passages to avoid runaway prompt size and reduce duplicate processing of the same sensitive text.

For a broader identity-and-secrets lens, the Ultimate Guide to NHIs is useful when context contains service-account references, API keys, or other secrets. The control also aligns conceptually with how NIST Cybersecurity Framework 2.0 emphasizes disciplined asset and data handling across systems.

Why It Matters in NHI Security

Context capping matters because AI agents often operate with privileged tools and reusable state, which can turn ordinary conversation history into an exposure path. If old instructions, secrets, or access tokens remain in active context, an agent may repeat them, act on stale assumptions, or surface data that should have aged out of the session. That is especially relevant where NHI controls already struggle: NHI Management Group reports that 96% of organisations store secrets outside of secrets managers in vulnerable locations, and 80% of identity breaches involved compromised non-human identities such as service accounts and API keys, as summarized in the Ultimate Guide to NHIs. In practice, context capping supports Zero Trust-style minimization by limiting what an agent can reuse from prior interactions, rather than assuming every remembered detail should remain actionable. It also reduces operational cost by preventing runaway prompt growth that can obscure auditability and increase failure rates. Organisationally, this control is easiest to overlook until a breach review shows that an AI assistant retained and replayed sensitive context after access should have been revoked. Organisations typically encounter this consequence only after a tool invocation or transcript leak, at which point context capping becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• What is the Model Context Protocol (MCP) and why does it matter for security?
• What is MCP in the context of AI security?
• When is it appropriate to implement MCP in the context of AI systems?
• Why is organizational context important for AI agents?