Notifications
Clear all
Topic starter
24/06/2026 9:08 pm
TL;DR: Internal RAG agents combine retrieval, prompt injection, and prompt compression in ways that improve usefulness but also enlarge the attack surface if retrieved content is not validated, sanitised, and constrained before it reaches the model, according to Kong. The real issue is that governance now extends into the retrieval path, where trust assumptions can be broken before the LLM ever responds.
NHIMG editorial — based on content published by Kong: Build Your Own Internal RAG Agent with Kong AI Gateway
By the numbers:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
- 96% of technology professionals identify AI agents as a growing security threat, and 66% believe this risk is immediate.
Questions worth separating out
Q: How should security teams govern retrieval paths in RAG systems?
A: Security teams should treat retrieval as part of the trusted control plane, not as a background search function.
Q: Why does prompt role choice matter in AI gateway design?
A: Prompt role choice matters because the role determines how much authority the model gives retrieved text.
Q: What breaks when compressed prompts remove security context?
A: When compression strips out provenance markers, policy text, or instruction boundaries, the model may still answer but with weaker governance semantics.
Practitioner guidance
- Classify retrieval sources by trust tier Separate internal documents, approved knowledge bases, and user-supplied content before they reach the vector store, and require different validation rules for each tier.
- Restrict privileged prompt roles Allow retrieved context to enter system-level prompts only when the source is trusted, governed, and auditable.
- Validate compressed prompts for policy loss Test the output of compression against a set of security-critical prompts to confirm that provenance markers, policy constraints, and instruction boundaries survive shortening.
What's in the full article
Kong's full blog post covers the operational detail this post intentionally leaves for the source:
- Step-by-step configuration for the AI RAG Injector plugin and prompt compressor settings
- Docker and Konnect setup details for the Redis vector database and compressor service
- Pre-function script instructions for ingesting chunks into the vector database
- Testing workflow that shows how compressed retrieval behaves in a live gateway setup
👉 Read Kong's guide to building an internal RAG agent with Kong AI Gateway →
Internal RAG agents and prompt injection: what IAM teams need to know?
Explore further
25/06/2026 6:09 am
RAG context injection is becoming a governance control, not a plumbing detail. Once retrieved content is allowed to shape model behaviour, the security question changes from whether the LLM is safe to whether the retrieval path is trustworthy. That means data source provenance, chunk validation, and role assignment all sit inside the control boundary. Practitioners should stop treating retrieval as a neutral pre-processing step.
A few things that frame the scale:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
- Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
A question worth separating out:
Q: How can organisations reduce prompt injection risk in RAG pipelines?
A: They should sanitize retrieved content, limit which sources can enter privileged prompt roles, and log the provenance of every chunk used in a response. Prompt injection is easiest to exploit when untrusted content is treated as authoritative context. The safest systems keep trust boundaries explicit and reviewable.
👉 Read our full editorial: Internal RAG agents expose the governance gap in AI gateway design
