RAG authorization at scale needs hierarchy, not per-document sync

At a glance

What this is: NHI Mgmt Group's analysis, drawing on WorkOS's blog post: Syncing every RAG document into the authorization layer does not scale, and hierarchical authorization with local filtering is the better pattern.

Why it matters: IAM and security teams need to know where access control should live in RAG architectures, because treating every chunk as a separately governed resource creates operational fragility and slows retrieval at scale.

Context

RAG authorization becomes a governance problem when the retrieval layer produces far more resources than the access model was designed to track. The article focuses on the mismatch between high-cardinality content stores and fine-grained authorization systems that expect stable resource sets, which is why the same control design that works in a demo fails under production load.

The practical question is not whether document access matters, but where the policy decision should sit. WorkOS argues that access should be governed at the collection, workspace, or knowledge-base level, then enforced locally in the vector store, so the authorization system does not become the bottleneck for the entire pipeline.

Key questions

Q: What breaks when every RAG document is synced into the authorization layer?

A: Per-document sync turns authorization into a throughput dependency. Every ingest, delete, and permission update has to hit the auth system, which creates latency, drift, and cleanup work that grows with the corpus instead of staying proportional to the policy problem.

Q: Why does per-document authorization create bottlenecks in RAG systems?

A: Because the auth layer has to process a much larger object set than the business access model actually requires. High-cardinality resources make every permission change more expensive, and that cost shows up as slower retrieval, operational fragility, and policy synchronization overhead.

Q: How should security teams govern access in RAG systems?

A: Security teams should govern RAG access at the retrieval layer, not only at authentication. That means mapping each workflow to the smallest possible set of collections, binding retrieval to user and service account entitlements, and separating sensitive corpora so one model path cannot reach unrelated business data. The goal is to limit blast radius before the model sees anything.

Q: When should organisations use document-level permissions in RAG?

A: Only when a document truly needs explicit control, such as shared files, special grants, or compliance holds. If document-level access becomes the default, the model becomes difficult to operate and audit, and it no longer reflects the normal inheritance pattern users actually follow.

Technical breakdown

Why per-document sync creates a high-cardinality authorization problem

Fine-grained authorization systems model relationships between identities and resources, but RAG transforms content into many more resources than most policy graphs are built to handle. One PDF can become dozens of chunks, a knowledge base can become millions of vectors, and each of those items may need lifecycle writes, deletion cleanup, and policy lookup. That creates operational fragility: latency enters the retrieval path, drift appears between systems, and the auth layer starts behaving like a processing queue rather than a policy engine.

Practical implication: Keep the authorization graph aligned to stable parent resources instead of forcing every retrieval artifact into the policy system.

How hierarchical authorization and local vector filtering separate policy from scale

The scalable pattern is to assign access at the collection, workspace, project, or folder level, then attach parent identifiers to vectors as metadata. At query time, the authorization system returns the set of parents a user can access, and the vector database applies that constraint locally. This preserves policy centralisation without requiring the auth system to inspect every document or chunk, which is the key architectural distinction in the article.

Practical implication: Use parent-level access decisions as the policy source and let the retrieval layer enforce the final filter on metadata.

When document-level permissions still make sense

Some RAG applications do need explicit per-document controls, especially for shared documents, exception-based grants, or compliance holds. The article treats that as an exception pattern, not the default operating model. That distinction matters because document-level governance is sustainable only when it is selective. If every object is exceptional, the hierarchy disappears and the scale problem returns in a different form.

Practical implication: Reserve document-level authorization for edge cases and keep the bulk of the corpus under hierarchical inheritance.

NHI Mgmt Group analysis

High-cardinality authorization is the real failure mode in RAG governance. The issue is not that authorization is missing, but that the object model explodes faster than the policy engine can comfortably track. When every chunk and embedding becomes a first-class entitlement target, the authorization layer inherits the scale burden of the retrieval system. Practitioners should treat resource cardinality as a governance design constraint, not an implementation detail.

Hierarchical access is the only sustainable control boundary for most enterprise RAG systems. Users usually need permission to a workspace, project, or collection, not to every derived retrieval artifact. That makes inheritance the correct control pattern, with local filtering handling the last-mile decision. The practical conclusion is that policy should stay stable while retrieval scales independently.

Document-level authorization should be an exception path, not the architecture baseline. Shared files, explicit grants, and compliance holds justify finer control, but only for the minority of cases that truly need them. If teams default to per-document sync for the entire corpus, they turn the auth layer into a throughput dependency and create a new class of operational drift. The discipline is to apply granularity selectively, not universally.

RAG authorization exposes a broader identity design principle: policy should attach to the least volatile resource that still preserves business meaning. That principle applies across human IAM, NHI governance, and retrieval architecture. Stable parent resources give you a control point that survives churn in downstream artefacts, which is why hierarchy is not just a scale optimisation but a governance model.

The named concept here is high-cardinality auth drift. It describes the gap that appears when access policy is forced to track rapidly multiplying retrieval objects instead of stable business containers. The implication is that governance breaks when entitlement management chases derived artefacts rather than the resource hierarchy the business actually recognises.

What this signals

High-cardinality auth drift: RAG systems should be governed around the least volatile resource that still reflects business access. When policy chases documents, chunks, and embeddings instead of collections or workspaces, control drift becomes structural and the auth layer starts competing with retrieval for runtime capacity.

Hierarchical authorization is not a compromise control. It is the correct separation of duties between policy decision and retrieval enforcement, and it maps cleanly to how most enterprises already think about access inheritance in content systems.

For practitioners

• Define the stable parent resources first Map collections, workspaces, projects, or knowledge bases before introducing document-level controls so the authorization model reflects the business hierarchy.
• Move access checks to query time Return the set of accessible parent resources once, then filter locally in the vector store using metadata tied to those parents.
• Treat document-level grants as exceptions Reserve per-document permissions for shared files, explicit overrides, or compliance holds so the default corpus stays under inherited access.
• Watch for authorization bottlenecks in ingestion Test whether high document-processing volume creates write latency, orphan cleanup work, or policy drift before the RAG pipeline reaches production.

Key takeaways

• Per-document authorization in RAG creates a scale problem because the number of derived retrieval objects grows faster than the governance model can track.
• The practical evidence in the article is operational fragility: ingestion writes, deletion cleanup, drift, and latency all move into the auth path when every chunk is synced.
• The sustainable pattern is hierarchical access at the parent-resource level, with local vector filtering handling the final enforcement step.

Key terms

• High-Cardinality Authorization Drift: The mismatch that appears when an authorization system is asked to govern far more resources than its policy model was designed to handle. In RAG, this usually means chunks, embeddings, and documents multiplying faster than entitlement state can stay accurate or operationally stable.
• Hierarchical Authorization: Hierarchical authorization assigns access to a stable parent resource such as a workspace, collection, or project, and lets child objects inherit that decision. It reduces policy volume, keeps revocation manageable, and matches how users usually understand access boundaries in enterprise systems.
• Local Vector Filtering: A retrieval pattern where the vector database applies access constraints using metadata at query time instead of relying on a separate authorization system to inspect each result. This keeps the policy decision small and stable while the retrieval layer performs the scale-sensitive filtering.
• High-Cardinality Resource: A high-cardinality resource is a large, fast-changing set of objects that can be created, updated, and deleted frequently. In authorization design, these resources are risky to synchronize externally because drift and latency can appear at the worst possible moment. Keeping them local can preserve speed and consistency.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.