Context Window Data Exposure: What Reaches Agent Prompts and How to Limit It

An AI agent's context window is assembled automatically from retrieval results, tool output and memory, so sensitive data reaches the model without anyone deciding it should. Every token in it can be repeated, logged, cached, sent to a provider or saved to memory. This guide treats the context window as a data flow that needs classification: what categories of data arrive from each source, which of them the task actually needs, and which controls filter, mask or block the rest before and after the model call. Harness architecture is covered in the AI Agent Harness and Context Engineering Security guide; this page is for data protection, AI platform and security engineers deciding what the agent is allowed to see.

Key takeaways

• Classify the flow, not just the stores. A document labelled Confidential in SharePoint loses its label once a retrieval step pastes it into a prompt, so the agent's context needs its own classification ceiling per task.
• Tool output needs minimising in the tool layer. Agents call APIs that return whole records, and guardrail filters are typically scoped to prompts and responses, so field allow lists in tool code decide what arrives.
• Memory turns a one-off exposure into a persistent one. Scan and classify before anything is written to long-term memory, and scope memory per user or tenant.
• Prompt inspection is a backstop, not the boundary. Pattern and classifier filters on prompts and responses catch known identifiers, but access control at retrieval decides what the model can see in the first place.
• Context leaves the window. Invocation logs, traces, caches and provider retention keep the original unmasked content, so data handling rules must follow the context into those stores.

Where sensitive data enters the context

Five sources feed an agent's context on a typical turn. Each has a different default for what arrives and who chose it. OWASP LLM02:2025 lists the categories at stake: personal data, financial details, health records, confidential business data, security credentials and legal documents. The table below maps them to the source that most often carries them in.

MITRE ATLAS describes deliberate abuse of this flow as AML.T0057 LLM Data Leakage (prompts that make the model reveal data from connected sources or other users) and AML.T0082 RAG Credential Harvesting (credentials in internal documents ingested into a RAG store and later retrieved). The EchoLeak case (CVE-2025-32711) shows both halves together: a crafted email was pulled into Microsoft 365 Copilot's context by retrieval, and injected instructions tried to move other data the user could reach out through a URL. Microsoft fixed it server side and reported no exploitation; the lesson is that whatever the agent can retrieve for a user is the exposure ceiling.

Build a classification ceiling for each agent

Apply the existing scheme (for example Public, Internal, Confidential, Restricted) to the agent's context rather than only to repositories. See the Data Classification Guide for the scheme itself.

1. Define the task's data need: for each agent, write down the highest classification and the specific data categories the task genuinely requires. A support triage agent may need a customer's name and order status; it does not need date of birth, card data or internal margin.
2. Set the ceiling: the ceiling is the maximum classification that may enter context for that agent and that caller. Anything above it is blocked or masked before inclusion.
3. Label each source: record the classification range each retrieval index, API and memory store can return. A vector index built from a mixed SharePoint estate has the classification of its most sensitive document unless it is partitioned.
4. Carry labels into context: keep the source label attached to each chunk or tool result as metadata the harness can read, so later steps (memory writes, summaries, outbound tools) can enforce rules on it.

The OWASP AI Agent Security Cheat Sheet gives a workable default for handling by tier: restricted data such as personal, financial and health data is fully redacted before context inclusion, and confidential data is partially masked. Treat credentials as restricted in every tier: they should never be in context at all. See Credential Vaulting for AI Agents for how to keep them out.

Controls at retrieval

Retrieval is where most over-exposure starts, because a ranking algorithm chooses content and the identity it runs under sets the limit.

• Run as the caller: Microsoft documents that Microsoft 365 Copilot only surfaces organisational data the user has at least view permission to, and that its Semantic Index honours the user's identity-based access boundary. Custom agents should do the same: query with the caller's token, not a service account that can read everything. See the Permission-Aware RAG Guide.
• Fix oversharing first: running as the caller is only as good as the permissions underneath. Microsoft Purview DSPM for AI, for example, runs a default data risk assessment weekly on the top 100 SharePoint sites by usage and offers remediation such as SharePoint Restricted Content Discovery, auto-labelling and removing sharing links.
• Exclude by label: Microsoft describes a Purview DLP policy that stops Microsoft 365 Copilot and agents from summarising items with selected sensitivity labels, and notes that for content reached through agents, encryption can exclude programmatic access. In custom pipelines, filter on label metadata at query time.
• Partition indexes: OWASP LLM08 recommends permission-aware vector stores with strict logical and access partitioning between user groups, and tagging and classifying data within the knowledge base. Build separate indexes for restricted collections rather than one index with a filter that can be forgotten. See the Vector Database Security Guide.
• Scan before ingestion: run secret scanning and sensitive information detection over documents before they are embedded, so credentials and identifiers never enter the index (the AML.T0082 pattern).

Minimising tool output

Tool output deserves its own section because it usually sits outside the scope of gateway inspection. The Amazon Bedrock Guardrails documentation, for example, scopes its sensitive information filter to input prompts and model responses and states that tool use content (model-generated tool arguments in toolUse.input, data the application returns in toolResult, and tool definitions) is handled by the application. Check the documented scope of whichever platform you use, and design the tool layer so that it minimises output itself.

• Field allow lists: each tool returns only the fields named in its schema. A get_customer tool for a delivery agent returns name, postcode and order status, not the full record. Drop fields in the tool code, not by asking the model to ignore them.
• Row limits and transport detail: cap result counts, and strip headers, stack traces and connection strings from errors before they reach the model.
• Tokenise identifiers: replace account numbers or national identifiers with reversible tokens the tool layer can resolve later, so the model reasons over CUST-7F3A rather than a real number.
• Inspect outbound arguments: the OWASP cheat sheet recommends checking tool-call parameters for sensitive values before execution, for example rejecting parameters that look like keys or passwords. Apply the same classification ceiling to data the agent sends out (email, webhooks, tickets), not only to data coming in.

Memory and history write filters

Long-term memory converts a single exposure into one that persists. The OWASP AI Agent Security Cheat Sheet recommends validating and cleaning content before persisting it, scoping memory per user and session, enforcing expiry and size caps, and scanning for sensitive data before saving. In practice:

• Classify at write time: run the same detectors used on prompts over every memory write, and refuse or redact anything above the agent's ceiling.
• Never store raw transcripts as memory: store distilled facts with a source reference, not the full turn that may contain pasted data.
• Scope and expire: per-user or per-tenant namespaces, a time to live on agent-written entries, and deletion that also removes embeddings.
• Support rights requests: if memory holds personal data, it is in scope for access and erasure requests under data protection law (for example GDPR), so it must be searchable by data subject.

Integrity threats to memory (injected instructions) are covered in the AI Agent Memory Security Guide; this section is only about confidentiality.

Prompt and response inspection

Inline inspection on the model gateway catches what minimisation misses. Cloud platforms ship this as a guardrail service; the examples below are listed alphabetically and are illustrations of configuration choices, not recommendations.

Decision guidance on actions:

• Mask rather than block on input when the task can proceed without the value (a name or email in a support query).
• Block credentials and regulated identifiers outright on input and output; there is no task where the model needs to repeat a secret key.
• Detect only first: start every new detector in a non-blocking mode for two to four weeks to measure false positives before enforcing.
• Remember the limits: OWASP LLM02 notes that system prompt instructions about which data types to return "may not always be honored" and can be bypassed by prompt injection. Pattern detectors also miss free-text sensitive content. Inspection reduces, it does not guarantee.

For user-facing GenAI DLP on public AI tools and endpoints, see the Generative AI Data Loss Prevention Guide.

Where context goes after the model call

Masking at the gateway does not clean the copies made elsewhere. The Bedrock documentation is explicit that model invocation logs in CloudWatch Logs and S3 always contain the original request and that guardrail trace output contains the original matched value. Microsoft states that Microsoft 365 Copilot stores the user's prompt and response with citations, subject to the organisation's retention policies.

How to implement

The phases below are NHI Mgmt Group recommendations.

1. Inventory (weeks 1 to 2): list each agent, its retrieval sources, tools, memory stores and log destinations, and the identity each runs as.
2. Set ceilings (weeks 2 to 3): agree a classification ceiling per agent with the data owner and record it.
3. Observe (weeks 3 to 6): enable prompt and response detection in non-blocking mode and sample logged context to measure what classes actually arrive from each source.
4. Fix the sources (weeks 4 to 10): switch retrieval to caller identity, remediate overshared sites, add field allow lists to the noisiest tools, add memory write filters.
5. Enforce (from week 8): move credential and regulated identifier detectors to block, and confidential types to mask, one agent at a time. Keep the previous guardrail version so a change that breaks a workflow can be reverted in minutes.
6. Clean downstream stores: apply log masking, trace redaction and retention limits; purge historic logs that hold restricted data.
7. Test: red-team prompts asking for data above the ceiling, and canary records in sources the agent should not reach, must both fail.

A small team using a single hosted assistant should concentrate on steps 1, 4 and 6: permissions hygiene and the vendor's built-in label and DLP controls deliver most of the value. Large estates with many custom agents need per-agent ceilings and a shared tool library with field allow lists, so minimisation is inherited.

Detection and measurement

• Above-ceiling rate: share of turns where content above the agent's classification ceiling reached context, from sampled context logs. Target falling to near zero.
• Detector hits by source: sensitive information detections broken down by user input, retrieval, tool output and memory, which shows where to fix upstream.
• Tool field coverage: percentage of agent tools with an explicit output field allow list.
• Caller-identity retrieval: percentage of retrieval calls made with the caller's identity rather than a shared service identity.
• Canary access: any retrieval of a seeded canary document is an alert and an investigation.

Common failure patterns

• Service account retrieval: the RAG pipeline indexes everything with a crawler identity and answers every user from it, so any employee can query HR or legal content through the agent.
• Guardrail on the prompt, nothing on tools: PII filters are configured on user input and model output while full customer records arrive through tool results that no layer minimises.
• Masked answer, raw logs: the user sees a masked response but invocation logs and traces keep the original, readable by a wide engineering group.
• Memory as transcript: the agent saves whole conversations as memory, so a card number pasted once is retrieved in later sessions.
• Labels lost in summaries: compaction or a sub-agent summary turns restricted text into an unlabelled paragraph that then flows to an outbound tool. The EchoLeak case shows how trusted data and untrusted instructions sharing one context lets data move to places the user never intended.
• Prompt rule as the control: "Never reveal personal data" in the system prompt is treated as the safeguard, which OWASP LLM02 and LLM07 both warn against.

Framework mapping

Practitioner checklist

• Every agent has a documented classification ceiling and list of permitted data categories.
• Retrieval runs with the caller's identity, and overshared sources were remediated before go-live.
• Restricted collections sit in separate, permission-partitioned indexes, scanned for secrets before ingestion.
• Every tool has an output field allow list and a row limit; identifiers are tokenised where the task allows.
• Memory writes pass through the same detectors as prompts, with per-user scope and expiry.
• Prompt and response inspection runs on input and output, with credentials blocked and confidential types masked.
• Invocation logs, traces and caches have masking, restricted access and retention matched to the highest class they hold.
• Canary records and above-ceiling red-team prompts run in every release test.

Standards and references

• OWASP: Top 10 for LLM Applications 2025 (LLM02, LLM07, LLM08)
• OWASP: AI Agent Security Cheat Sheet
• MITRE ATLAS: LLM Data Leakage (AML.T0057)
• NIST: SP 800-53 Rev. 5
• NIST: Cybersecurity Framework 2.0
• AWS: Sensitive information filters in Amazon Bedrock Guardrails
• Google Cloud: Model Armor overview
• Microsoft Learn: Data, privacy and security for Microsoft 365 Copilot

Related NHI Mgmt Group resources: AI Agent Harness and Context Engineering Security · OWASP LLM Top 10 LLM02 Sensitive Information Disclosure Guide · Generative AI Data Loss Prevention Guide · Permission-Aware RAG Guide · Data Minimisation