How should teams place access controls in a two-stage RAG pipeline?

Why control placement matters in a two-stage RAG pipeline

A two-stage rag pipeline creates a distinct policy boundary between retrieval and ranking. Candidate generation is the point where the system first collects possibly relevant material, while reranking is the point where the system decides what is most worth surfacing to the model. If access controls are enforced before reranking, the system can prevent disallowed content from ever being promoted into the final context set.

That placement matters because reranking is not a neutral step. It can amplify a sensitive document, source, or snippet into the top position even when the content should never be considered for the user or session. In practice, the control should act on the candidate pool, not on the already-ranked shortlist.

For teams building permission-aware retrieval, the practical question is not whether the retriever can find the content, but whether the pipeline can still stop it from becoming a candidate for exposure. NHIMG’s Permission-Aware RAG Guide frames that problem as a retrieval-time authorization issue, which is the right mental model for this control point.

What the filter should see, and what it should block

The filter should evaluate every candidate returned by retrieval against the user’s entitlement, data classification, tenancy boundary, and any document-level policy attached to the source. That can include row-level permissions, ACLs, group membership, contextual policy, or ownership rules, depending on how the corpus is governed. The key is that the control is applied before the reranker sees the candidate as eligible for promotion.

In a well-designed pipeline, the reranker only scores items that have already passed policy. That keeps the ranking step focused on relevance rather than on deciding whether a source is even allowed to participate. It also reduces the chance that a sensitive but highly similar item appears to be the “best” answer path simply because it matches the query well.

The same design principle appears in broader authorization guidance: choose the policy model that matches the resource boundary, then enforce it at the point where candidate access is decided. NHIMG’s Authorisation Models Guide is useful here because it distinguishes role-based, attribute-based, and relationship-based approaches that can all feed a retrieval-time decision.

For teams that still treat RAG controls as an application-layer afterthought, this is where the design fails. Access rules that live only after reranking are too late to shape what the model can see. NHIMG’s Broken Access Control Prevention Guide is relevant because the same server-side principle applies: enforce the decision before the protected object can influence output.

How to keep the pipeline safe when a control cannot be applied earlier

If policy cannot be enforced directly at retrieval time, the fallback is to quarantine the candidate set and ensure reranking still runs only on authorised material. That may mean a pre-rerank policy gate, a post-retrieval entitlement check, or a filtered index design that never exposes disallowed items to the ranking service. What matters is that the control sits before any step that can elevate a candidate into context.

Teams should also assume that indexes, embeddings, and vector stores can drift away from the source-of-truth permission model. A document may remain physically retrievable even after access should have changed, so the permission signal must be checked against current policy rather than inherited from ingestion time. NHIMG’s RAG Security Guide is a good companion reference for source trust, retrieval poisoning, and vector-store exposure risks.

Where retrieval spans multiple tenants or business units, the safest pattern is to make the policy decision on the candidate set itself, not on the final prompt. That reduces blast radius if a source is misclassified, an index is over-broad, or a query is crafted to pull in an otherwise hidden item. It also gives you a clearer audit trail for why a candidate was excluded before ranking.

Risk and Threat Considerations

Late-stage access control creates a real exposure window because the reranker may already have promoted a sensitive document, snippet, or embedding match into the path the model will prefer. That can turn a simple entitlement failure into data exposure, over-sharing, or policy bypass at context-building time.

Failure mechanism: The candidate set is scored before policy is enforced, so an unauthorised item can be ranked highly, preserved in the shortlist, and then surface as part of the model’s available context even though access should have been denied.

Impact: Sensitive content can leak into the answer path, the system can expose cross-tenant or out-of-scope material, and incident response becomes harder because the access failure happened before the final generation step.

Practitioner Guidance

What to verify: Confirm that the access decision is made on the retrieved candidate set before reranking, and that the reranker never receives items that policy would later reject. Validate this with tests for denied users, cross-tenant queries, and document-level permission changes.

Common mistake: Teams often place controls at the prompt assembly stage because that is where leakage is most visible. By then, the damage is already done, because ranking and selection have already shaped what the model is likely to use.

Practitioner takeaway: Treat retrieval-time filtering as the security control and reranking as a downstream relevance function, not the other way around. If policy is not decided before ranking, the pipeline can still “choose” a forbidden answer path.

Related resources from NHI Mgmt Group

• How should security teams decide whether JIT access is safe for non-human identities?
• How should teams enforce access controls in RAG applications that merge internal datasets with LLMs?
• How should security teams run vulnerability scans on applications protected by two-factor authentication without breaking access controls?
• Should teams treat prompt design or pipeline controls as the main safeguard for long-document RAG?