What breaks when prompt injection controls only inspect user prompts and not retrieved content?

Why This Matters for Security Teams

Prompt injection is not limited to what a user types. In retrieval-augmented systems, the unsafe instruction often lives inside a PDF, wiki page, email thread, ticket, source file, or metadata field that the model later consumes as trusted context. When controls inspect only the chat prompt, they miss the real attack path: retrieved content that can steer reasoning, leak secrets, or provoke tool use. The OWASP Agentic AI Top 10 treats this as a core design flaw, not an edge case.

This matters because the model does not distinguish between “user intent” and “retrieved instruction” unless the architecture forces that separation. Attackers exploit that gap by planting malicious text where indexing, chunking, or metadata extraction will surface it later. NHIMG has documented the operational reality in incidents such as the Gemini AI Breach — Google Calendar Prompt Injection, where hidden instructions in non-chat content influenced downstream behaviour. In practice, many security teams discover this only after a retrieval path has already exposed data or triggered an unsafe action, rather than through intentional testing.

How It Works in Practice

Effective protection has to cover the full retrieval and ingestion path, not just the final prompt. That means treating retrieved content as untrusted input, even when it comes from an internal repository or “approved” knowledge source. A practical control stack usually includes content scanning before indexing, sanitisation of retrieved chunks, instruction-detection heuristics, and runtime policy checks before any tool call or sensitive response.

At minimum, teams should separate system instructions, user input, and retrieved context so the model cannot freely merge them. They should also log the provenance of every retrieved segment so suspicious outputs can be traced back to the exact source document, chunk, or metadata record. Where the system can act on behalf of a user, OWASP Agentic Applications Top 10 and the OWASP Agentic AI guidance both point toward layered defences rather than a single “prompt firewall.”

• Scan retrieved text for instruction-like phrasing, credential requests, and tool-triggering language before it reaches the model.
• Block or rewrite risky metadata, comments, and hidden fields during ingestion, not only at query time.
• Apply retrieval allowlists so only expected sources and data classes can enter the context window.
• Use output and action guards so a malicious retrieval cannot directly trigger file access, email sending, or secret exfiltration.

NHIMG’s research on the Ultimate Guide to NHIs shows why this matters operationally: secrets exposure and over-privileged access are common, so a single compromised retrieval path can turn into an identity and access incident as well as an AI safety issue. These controls tend to break down when retrieved content arrives from highly fragmented sources such as indexed chat exports, embedded document fields, or third-party connectors because provenance and trust boundaries become ambiguous.

Common Variations and Edge Cases

Tighter retrieval filtering often increases friction, requiring organisations to balance prompt-injection resistance against recall, usability, and support overhead. That tradeoff becomes obvious in systems that must search large internal corpora, where aggressive filtering can remove legitimate instructions alongside malicious ones.

Best practice is still evolving for edge cases such as code repositories, meeting transcripts, and structured records. A code comment that looks like an instruction may be harmless in one workflow and dangerous in another. Likewise, a retrieval pipeline that sanitises prose may still pass through risky tokens in JSON fields, file names, or embedded markdown. Current guidance suggests context-aware handling: classify the source, assign trust levels, and apply different rules for human-authored notes, machine-generated artifacts, and externally supplied content.

For teams building agentic workflows, the main failure mode is assuming that retrieval is “read only.” In reality, retrieved instructions can change tool selection, reorder reasoning, or prompt the model to reveal data that should never have entered the context window. The most reliable pattern is to validate both what is retrieved and what the model is allowed to do with it, using policies that evaluate at runtime rather than fixed prompt filters alone.

Related resources from NHI Mgmt Group

• What breaks when prompt injection and jailbreak controls are only added in application code?
• What breaks when AI guardrails only focus on toxic content and prompt injection?
• What breaks when prompt injection controls focus only on the prompt plane?
• What breaks when prompt injection controls are not tested against real attack patterns?