Model Context Leakage

What Model Context Leakage Actually Means

Model context leakage is broader than a single prompt being exposed. The risk is that any working context, including retrieved documents, tool results, prior conversation state, or hidden instructions, can escape the intended boundary and become visible to the wrong party or session.

That matters because context is often where the most sensitive material sits: customer data, internal strategy, system prompts, credentials embedded in examples, or task details that were never meant to be persisted. Once leaked, the model may still behave “correctly” from its own perspective while quietly disclosing information that should have remained scoped.

How Context Leaks Happen in AI Systems

Leakage usually comes from boundary failures rather than one single defect. Logging pipelines may capture raw prompts and responses, orchestration layers may forward data to the wrong tool, and weak session isolation can let one user inherit another user’s context. Prompt injection can also cause a model or agent to reveal or relay material it was only supposed to use internally.

In agentic and retrieval-augmented systems, the problem gets harder because context is distributed across model memory, vector stores, plugins, browsers, connectors, and middleware. The more places context is copied, transformed, or replayed, the more opportunities there are for accidental disclosure or cross-tenant contamination.

Why Context Leakage Is Security-Relevant

Leaked context can expose far more than a single answer. It can reveal private source data, internal workflows, hidden prompts, policy rules, and the existence of sensitive integrations. If the context includes secrets or session artefacts, the leak can become an access problem as well as a confidentiality problem.

This is why context leakage is often tied to secondary harms such as impersonation, unauthorized tool use, replay of privileged instructions, or reconstruction of internal knowledge that was never meant for external consumption. The security issue is not only disclosure, but also the trust boundary collapse that follows it.

Tools that already reason about AI attack paths can help frame this risk. NHIMG’s OWASP Agentic Applications Top 10 and the OWASP Agentic AI Top 10 both provide useful structure for understanding prompt injection, tool misuse, and identity or privilege abuse around agentic systems.

How Teams Reduce Exposure and Verify Containment

The practical control problem is to keep context scoped to the minimum necessary boundary and to verify that it is not copied into places where it should not persist. That includes careful logging rules, strict session separation, filtered tool outputs, and explicit handling of retrieved content that may contain sensitive material.

Teams should also review how prompts and tool responses move through gateways, apps, and orchestration layers. If the system depends on external retrieval or MCP-style tool chains, the context boundary must be treated as an enforcement point, not just an implementation detail. NHIMG’s MCP Security Guide, MCP Apps Security Guide, and MCP Gateway Buyers Guide are relevant where authorization, gateway policy, and tool visibility determine whether context can cross an unsafe boundary.

For readers who want the official protocol-side view, the Model Context Protocol: Authorization specification is a useful external reference for audience-bound tokens and no token passthrough, while the MITRE ATLAS adversarial AI threat matrix helps place context poisoning and related AI attack techniques in a broader threat model.

Detecting Leakage After the Fact

Leakage often becomes visible only after the system has already forwarded or emitted the wrong context, so detection needs to focus on traceability. That means retaining enough audit detail to reconstruct which prompt, retrieved item, tool result, or session state was involved, while still avoiding unnecessary exposure in the logs themselves.

Post-incident review should ask whether a model disclosed data it had access to, whether the disclosure crossed tenant or session boundaries, and whether any downstream component stored or replayed the compromised context. The question is not only what leaked, but where the leak originated and whether the same path can be reused.

When the leak intersects with a broader compromise campaign, threat intelligence can help distinguish accidental exposure from active exploitation. Anthropic’s report on the first AI-orchestrated cyber espionage campaign offers a useful external example of how AI-assisted operations can combine reconnaissance, credential harvesting, and exfiltration into a single chain of abuse.

Related resources from NHI Mgmt Group

• Model-context Secret Leakage
• What is the Model Context Protocol (MCP) and why does it matter for security?
• How should security teams govern AI agents that use Model Context Protocol?
• Why does Model Context Protocol create identity risk for enterprises?