Should organisations keep access control inside the LLM or outside it?

What belongs outside the LLM, and why?

Access control belongs outside the model because the model should not be the final authority on who can see, retrieve, or act on protected data. The safer pattern is to let external policy decide what context is allowed, then pass only governed material into the LLM. That keeps authorization deterministic, auditable, and easier to change without retraining or prompt rewrites.

This is especially important when retrieval, connectors, vaults, or tool calls can expand the model’s reach. A model can summarise, rank, or transform content, but it should not independently decide whether a user may receive a record, a secret, or a privileged action. In practice, permission-aware RAG is the clearest example of the pattern: retrieve only what the requesting identity is already allowed to see, then let the LLM work inside that boundary.

That separation also applies to the systems that feed the model. The access decision should happen before content reaches the prompt, before a secret is exposed to a tool, and before a retrieval layer returns cross-tenant or cross-role material. When organisations treat the model itself as the access control point, they create a soft policy layer that is harder to test, harder to monitor, and easier to bypass through prompt leakage or over-broad context assembly.

Why in-model access control creates a bigger blast radius

Putting authorization inside the LLM collapses two different jobs into one: reasoning about language and enforcing security policy. That sounds convenient, but it means a prompt injection, malformed instruction, or model mistake can become a privilege decision. The same issue appears in workflows that mix context, tools, and secrets, where the model is asked to decide both relevance and entitlement.

The better architecture is to keep the model as a consumer of governed context, while external controls enforce least privilege, secret scoping, and retrieval filtering. If the model never receives data it should not see, the failure mode stays smaller. If it does receive it and then leaks it in output, the damage is still serious, but the control failure is visible at the policy boundary instead of buried inside generation logic.

For AI platforms that use credentials, APIs, or retrieval over indexed content, the same principle is reinforced by AI Infrastructure Workload Identity Guide and LLM Provider API Key Security and LLMjacking Guide: access boundaries belong in the surrounding control plane, not in the language model itself.

What a safer access pattern looks like in practice

A practical design starts with three controls: policy enforcement outside the LLM, governed retrieval, and tightly scoped secrets handling. The model should receive only the minimum context needed for the task, with the caller’s permissions checked before retrieval and before any tool invocation. Where a response needs data from multiple sources, each source should be filtered independently rather than merged first and filtered later.

• Enforce user and application permissions in the gateway, retrieval layer, vault, or tool broker.
• Keep sensitive values out of prompts unless they are strictly needed for the task and already authorised.
• Use short-lived, scoped tokens or delegated access paths for tool calls instead of embedding standing credentials in model context.
• Log the policy decision, the retrieved objects, and the tool action so access can be reviewed after the fact.

This is also why agentic systems need special care. Once a model can choose tools or chain actions, the question is no longer just “what text was generated?” but “what authority was exercised?” The safest pattern is to authorise each action externally and treat the model’s output as a request, not as permission. Agentic AI Security Guide and Enterprise AI Copilot Security Guide both support that design choice.

Risk and Threat Considerations

When access control lives inside the LLM, a prompt leak can turn into a privilege leak. The threat is not only accidental disclosure, but also prompt injection, overshared retrieval, and tool abuse that cause the model to reveal or act on material the caller should never have reached. The larger the context window and the broader the connectors, the easier it is for one weak decision to expand into multiple exposures.

Failure mechanism: The model is asked to infer authorization from language context instead of enforcing policy at a deterministic boundary, so an attacker can manipulate instructions, retrieval, or tool selection to cross the intended access line.

Impact: Sensitive records, secrets, or privileged operations can be exposed through the prompt path, and the organisation loses clear separation between content generation and access enforcement.

Practitioner Guidance

What to verify: Check whether every retrieval source, connector, vault lookup, and tool call is evaluated against external policy before the LLM sees the result. If any sensitive object is filtered only after prompt assembly, treat that as a design defect rather than a tuning issue.

Decision rule: If the control must prevent disclosure, do not put it in the model. If the control only shapes wording or summary style, it may belong in the prompt layer, but the entitlement decision should still be made elsewhere.

Common mistake: Teams often assume that hidden system prompts or “trusted” model behaviour are enough to protect access. In practice, that is a content-control assumption, not an authorization control, and it does not scale well when connectors, agents, or shared retrieval are added.

Practitioner takeaway: Treat the LLM as a governed consumer of context, not as the system of record for access decisions. The more an AI workflow can retrieve, call, or act, the more important it becomes to keep authorization outside the model and make the boundary observable.

Related resources from NHI Mgmt Group

• How can organisations keep directory-based access under control?
• How do organisations keep MCP session state from becoming an access-control blind spot?
• How do organisations keep AI features from weakening privacy and access control?
• Which IAM control matters most when organisations need to keep access available during identity provider outages?