They often focus on the model and ignore the retrieval layer. RAG attacks succeed when malicious or over-sensitive content is allowed into context, so the real control point is document selection, ranking, and sensitivity filtering before generation starts. Static access rules alone do not capture that interaction risk.
Why This Matters for Security Teams
Security teams often treat RAG as a model safety problem, then discover the real exposure sits upstream in retrieval. The failure mode is not only prompt injection. It is also malformed document ranking, stale content, over-broad indexing, and sensitivity leaks that place harmful context into the generation step before the model ever reasons about it. That makes RAG risk a data-access and control-plane issue as much as an AI issue, which is why the NIST Cybersecurity Framework 2.0 remains relevant: identify, protect, detect, and govern the retrieval layer, not just the output layer.NHIMG guidance on Top 10 NHI Issues and the OWASP NHI Top 10 both point to the same operational truth: hidden identity and access paths often become the attack path. In RAG systems, those paths include connectors, embeddings pipelines, chunk stores, vector databases, and search filters. In practice, many security teams encounter hallucination blame after the breach has already been caused by unsafe retrieval and exposure of context.
How It Works in Practice
Effective rag security starts before the model sees any context. The retrieval pipeline should enforce document-level authorization, tenant boundaries, freshness rules, and content classification before ranking occurs. If indexing is broad but retrieval is narrow, the system still needs pre-generation filtering because ranked results can surface sensitive material even when the final answer is policy-restricted.Practitioners should think in terms of controls, not just prompts:
- Restrict source connectors to approved systems and approved scopes.
- Filter documents by sensitivity, business context, and user entitlement before retrieval.
- Log which chunks were selected, why they were selected, and which policy allowed them.
- Use separate controls for ingestion, indexing, retrieval, and answer generation.
- Monitor for prompt injection in source content and malicious instructions embedded in retrieved text.
This is also where identity matters. Service accounts, API keys, and retrieval workers need least privilege because they can expose content indirectly even when the end user should not see it. The Ultimate Guide to NHIs — Key Challenges and Risks frames this well: control of non-human access paths is now part of application security, not just infrastructure hygiene. For implementation detail, current guidance from OWASP and NIST aligns with pre-execution content filtering, strong telemetry, and context-aware authorization at request time. These controls tend to break down when organizations index unvetted shared drives or third-party knowledge bases because access rules at query time cannot undo unsafe material already embedded in the corpus.
Common Variations and Edge Cases
Tighter retrieval controls often increase operational overhead, requiring organisations to balance precision against search utility and support burden. That tradeoff is real, especially in customer support, internal knowledge search, and regulated environments where teams want broad recall but cannot tolerate accidental disclosure.There is no universal standard for this yet, but current guidance suggests three edge cases deserve special handling. First, multi-tenant RAG needs tenant-isolated indexes or very strong retrieval-time partitioning; shared embeddings with weak filters are a common failure pattern. Second, hybrid search can reintroduce risk if keyword ranking bypasses sensitivity checks that were applied to vector similarity results. Third, highly dynamic content, such as incident notes or collaborative docs, can become unsafe between ingestion and retrieval unless refresh and reclassification are continuous.
For teams benchmarking maturity, the State of Non-Human Identity Security is a useful reminder that visibility gaps are common: 85% of organisations lack full visibility into third-party vendors connected via OAuth apps. While that stat is about NHI more broadly, the lesson transfers directly to RAG because hidden upstream access paths often determine what retrieval can reach. The practical answer is to govern the content supply chain, not just the model boundary.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A03 | RAG risk often begins with injected or malicious context entering the model. |
| CSA MAESTRO | M2 | MAESTRO addresses agent and tool governance that also applies to RAG pipelines. |
| NIST AI RMF | AI RMF covers governing and measuring retrieval-related AI risk. |
Treat retrieval controls as part of AI governance, with monitoring, accountability, and documented risk decisions.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on June 11, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
