How do security teams decide whether to centralise LLM authentication in the gateway or keep it inside each agent?

Why This Matters for Security Teams

The decision is not just where a login happens. It determines whether authentication is governed as a shared control plane or scattered across agent code, where drift, secret sprawl, and inconsistent policy are much harder to manage. For AI agents, that matters because their actions are goal-driven and runtime-dependent, so access patterns can change from one task to the next. Guidance from OWASP Agentic AI Top 10 and the NIST AI Risk Management Framework both point security teams toward runtime governance, not static trust assumptions.

NHIMG research shows why this choice is no longer theoretical: in AI Agents: The New Attack Surface, 80% of organisations said their AI agents had already acted beyond intended scope, including accessing unauthorised systems and exposing credentials. That is a strong signal that auth placement affects blast radius, auditability, and incident response. In practice, many security teams discover the weakness only after an agent has already reused a token or called a downstream tool in an unexpected chain, rather than through intentional design review.

How It Works in Practice

In enterprise designs, centralising authentication in the gateway usually means the gateway handles token issuance, session validation, policy checks, logging, and revocation. The agent then receives only a narrow, short-lived token or delegated assertion for the specific task. That reduces credential exposure in prompts, code, and tool wrappers, and it makes rotation far easier. This model also fits better with CSA MAESTRO agentic AI threat modeling framework and NHI research such as The State of Non-Human Identity Security, which highlights poor rotation and weak visibility as common failure modes.

Keeping auth inside each agent can work in prototypes, but it shifts policy enforcement into many places. That often creates inconsistent scope checks, duplicate secret storage, and uneven audit logs. A better pattern is to pair the gateway with workload identity and runtime authorization:

• Use workload identity for the agent, not shared static secrets.
• Issue short-lived credentials per task or per tool call where possible.
• Evaluate policy at request time, based on task context, data sensitivity, and tool risk.
• Log both the auth decision and the downstream action so incident response can reconstruct chain-of-custody.

For teams implementing this pattern, standards like NIST AI Risk Management Framework and research on OWASP NHI Top 10 both support minimizing standing privilege and reducing where secrets live. These controls tend to break down when agents must operate offline or across fragmented legacy systems because runtime policy and central token services are then harder to reach consistently.

Common Variations and Edge Cases

Tighter gateway control often increases orchestration overhead, so organisations have to balance simplicity against latency, service coupling, and developer autonomy. That tradeoff is real in hybrid environments, regulated data zones, and multi-agent workflows where one agent delegates to another. Current guidance suggests centralising auth for shared enterprise tools, while allowing narrowly bounded local handling only when the gateway cannot reliably mediate the interaction.

There is no universal standard for this yet, but the practical split is becoming clearer. Centralise when the agent touches high-value systems, regulated data, or cross-domain tools. Keep auth closer to the agent only when the scope is tiny, the blast radius is low, and the agent is effectively a scripted workflow rather than an autonomous system. Security teams should also treat any design that embeds long-lived API keys in agent code as a red flag, especially when the same agent can chain tools or escalate scope. NHIMG’s coverage of agentic risks, including CoPhish OAuth Token Theft via Copilot Studio, shows how quickly local convenience becomes a credential-exposure problem.

Where teams need a decision rule, the safest default is to centralise auth in the gateway unless the agent is isolated, single-purpose, and not allowed to make new authorisation decisions at runtime. That approach aligns with the OWASP Agentic AI Top 10 and the MITRE ATLAS adversarial AI threat matrix, both of which treat uncontrolled tool use and privilege escalation as core risks.

Related resources from NHI Mgmt Group

• How should security teams decide whether JIT access is safe for non-human identities?
• How do security teams decide whether an AI agent should keep access to regulated data?
• How do security teams decide whether to compare gateway-based governance with point controls around each agent or tool?
• How do security and platform teams decide whether to centralise AI agent evaluation across multiple build paths?