RAG policy fails when organisations treat query permission as the whole control and ignore what retrieval returns into context. The answer can be authorised at the door and still expose sensitive details inside the corpus. Effective governance requires both ingestion-time protection and retrieval-time filtering so the model only sees the minimum content needed for the task.
Why RAG policy fails at the retrieval boundary
RAG policy usually fails when it is written as a gate on the user query, not as a control on what retrieval is allowed to place into the model’s context. If the corpus contains sensitive material, a fully permitted request can still surface it through search, chunk selection, or ranking. That makes retrieval-time filtering and document-level permissioning part of the control, not an afterthought.
The practical mistake is assuming the model only needs to answer whether the prompt is allowed. In RAG, the real exposure is often created before generation starts, when the system assembles context from indexed data, embeddings, vector stores, or connected repositories. If those sources are not filtered to the requester’s rights, the model can disclose content that the user should never have been able to see.
RAG policies also fail when they treat “relevance” as a substitute for “authorization.” A document can be highly relevant to the question and still be off-limits because it contains secrets, personal data, internal plans, or other restricted material. The answer quality may improve while the security posture worsens, which is why retrieval rules must account for sensitivity, not just semantic similarity.
What must be controlled before sensitive data reaches the model
The core control point is the pipeline that decides which chunks, records, or passages are eligible for retrieval. That includes ingestion-time classification, metadata retention, access labels, filtering rules, and any permissions carried from the source system into the index or vector store. If those source permissions are lost during indexing, the model may inherit a broad read path that no individual user actually has.
A second control point is the retrieval layer itself. Even when data is safely indexed, the system should still enforce per-user or per-role filtering at query time so the returned context matches the requester’s entitlement. For many deployments, the safest pattern is to combine document-level permissions with sensitivity-aware redaction or suppression before the model ever sees the text.
Minimum necessary context matters here. The more the model sees, the more likely it is to echo content that is operationally sensitive but unnecessary for the task. A useful RAG policy therefore limits the scope of retrieved passages, trims oversized context, and blocks categories that should never enter the prompt, such as secrets, credentials, regulated personal data, or privileged internal deliberations.
Why governance has to cover indexes, vector stores, and source systems
RAG leakage often begins upstream of the model, in the systems that ingest and store source material. If the index contains over-shared files, stale permissions, or weakly governed repositories, the retrieval layer can faithfully reproduce a data problem the organisation already had. The right question is not only “who can ask?”, but also “what is now searchable, retrievable, and recombinable?”
That is why governance has to cover the source system, the indexing process, and the retrieval service as one chain. Sensitive data should be excluded, masked, or partitioned before indexing where possible, and access controls should remain attached to the content through its lifecycle. When the data is especially sensitive, treat the vector store and any derived artifacts as governed assets, not neutral technical plumbing.
RAG policy also needs retention discipline. If obsolete or duplicated content remains indexed, the model may surface material that should already have been expired, removed, or reclassified. A clean retrieval boundary depends on current data handling, not just a one-time policy statement.
Risk and Threat Considerations
When retrieval is more permissive than the user’s actual access, RAG can become a disclosure path even if the application front door is properly authenticated. The risk is not limited to accidental leakage, because an attacker or curious insider can deliberately ask questions that cause the system to surface restricted passages, then use the model’s summarisation to extract what they could not directly browse.
Failure mechanism: Retrieval returns semantically related but unauthorised content into the prompt, and the model converts that content into a readable answer, summary, or paraphrase. The control fails because permission checks stop at query entry instead of governing retrieval eligibility and context assembly.
Impact: Sensitive business data, personal data, secrets, or privileged internal material can be exposed through a system that appears to be operating within its normal access model. That can create confidentiality breaches, compliance issues, and downstream misuse of whatever the model reveals.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP API Security Top 10 addresses the attack surface, NIST SP 800-53 Rev 5 sets the technical controls, and ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP API Security Top 10 | API5 — Broken Function Level Authorization | RAG retrieval needs authorization beyond the user query itself. |
| Recommendation — Enforce function-level authorization on retrieval endpoints and context assembly. | ||
| NIST SP 800-53 Rev 5 | AC-6 — Least Privilege | RAG should only retrieve the minimum content a requester needs. |
| IA-2 — Identification and Authentication (Organizational Users) | User identity is the basis for enforcing retrieval entitlements in enterprise RAG. | |
| AC-3 — Access Enforcement | Retrieval must enforce content access rules, not just accept queries. | |
| Recommendation — Limit retrieval and context assembly to the minimum privileged data required. Authenticate users before applying retrieval entitlements and content filters. Apply access enforcement at retrieval time so unauthorized content never reaches context. | ||
| ISO/IEC 27001:2022 | A.5.15 — Access control | RAG policy must define who can retrieve which content and under what conditions. |
| Recommendation — Define and enforce access control for indexed content and retrieval outputs. | ||
Practitioner Guidance
What to verify: Test the full path, not just the chat front end. A request should only retrieve content that the same user could legitimately access in the source system, and the test should cover indexing, retrieval, reranking, and final prompt assembly.
Decision rule: If a control only checks whether the user may submit a question, it is incomplete for RAG. Treat retrieval filtering and content minimisation as mandatory when the corpus contains anything sensitive enough that a paraphrased answer would still be harmful.
Common mistake: Teams often validate that the model will not answer forbidden questions, then miss the more important failure mode where an allowed question pulls back restricted evidence. In RAG, the security boundary is the context window, not just the prompt box.
Practitioner takeaway: Effective RAG governance is about controlling what enters context, because once sensitive material is retrieved, the model can transform a permitted query into an unauthorised disclosure.
Related resources from NHI Mgmt Group
- Guide: Permission-Aware RAG Guide
- Why do traditional access controls fail to protect sensitive data in cloud and AI environments?
- Why do IAM controls fail when sensitive data spreads across cloud storage and AI workflows?
- Why do privacy workflows fail when sensitive data is spread across cloud and AI environments?
- Why do legacy DLP controls fail when sensitive data becomes fragmented across collaboration and AI workflows?