TL;DR: AI gateways are converging around three jobs, model routing, MCP tool mediation, and broader agent communication, but the article’s key finding is that routing alone does not answer who is authorised to act, according to Aembit. That boundary is governed upstream by attested workload identity, represented-user context, and policy-driven credential issuance, not by the gateway label itself.
At a glance
What this is: This article explains why AI gateways can route and mediate traffic, but cannot by themselves decide who has authority to act.
Why it matters: For IAM teams, the boundary between routing and authorisation matters because agentic systems need workload identity, delegated context, and credential issuance to be governed before any gateway enforcement can be trusted.
Context
AI gateways sit at the boundary between an application, model, tool, or agent and the systems it is trying to reach. In agentic environments, that boundary is more crowded than in traditional API architectures because requests can move across models, MCP servers, SaaS platforms, databases, and other agents.
The security problem is not just where to route a request, but which identity and authority travel with it. That makes AI gateway design an identity governance question as much as a traffic-management question, especially when multiple actors and delegation paths are involved.
The article separates gateway functions from access decisions. Routing can make agentic systems operable, but it does not by itself answer who may act, on whose behalf, or with which credential.
Key questions
Q: How should teams govern AI gateways that route model and tool traffic?
A: Teams should treat the gateway as the control boundary for identity, spend, logging, and policy enforcement. That means registering each AI workload or agent, attaching a clear owner, and ensuring every significant model or tool call is traceable. The goal is not to slow AI down, but to make it accountable in production.
Q: Why does gateway routing not answer who is allowed to act in agentic systems?
A: Routing decides where traffic goes, not whether the caller has delegated authority to take the action. In agentic systems, the meaningful security question is which workload is acting, which human or organisational identity it represents, and which credential was issued for the approved task. Without that context, a gateway can move requests but cannot govern authority reliably.
Q: Where do AI gateways fail if identity context is missing?
A: They fail at the point where enforcement depends on knowing more than the destination. A gateway can authenticate a connection and still be blind to represented-user context, policy scope, or destination credential provenance. In that case it becomes a traffic router with partial visibility, which is insufficient for delegated access in agentic environments.
Q: What is the difference between an MCP gateway and an LLM gateway?
A: An MCP gateway manages how agents connect to tools and data sources, while an LLM gateway manages how applications connect to the model itself. LLM gateways usually focus on routing, rate limiting, and token optimization. MCP gateways focus on tool discovery, authorization, tracing, and controlling what an agent can do once it reaches enterprise systems.
Technical breakdown
What an LLM gateway controls, and what it does not
An LLM gateway, sometimes called a model gateway, manages inference traffic across model providers and deployments. Its core jobs are routing, retries, fallback, rate limits, caching, token accounting, cost controls, logging, and sometimes content inspection. That makes it an operational control plane for model consumption, not an authority plane for downstream actions. If an agent receives a tool call from a model, the gateway can help decide which model answered. It cannot, on its own, establish whether the agent is entitled to execute that tool call or with what delegated authority.
Practical implication: Separate model-routing controls from authorisation controls so model selection never substitutes for access approval.
How MCP gateways mediate tool access
An MCP gateway sits between MCP clients and MCP servers to consolidate discovery, route requests, apply policies, authenticate connections, and record activity. The Model Context Protocol now gives protected servers an OAuth 2.1-style foundation, including authorization-server hardening in newer specifications. That improves protocol-level access handling, but it still leaves enterprise questions unresolved: how to attest the calling workload, preserve represented-user context, and broker the destination credential needed by the tool. In other words, MCP standardisation helps with transport and protocol trust, but not with full delegation governance.
Practical implication: Treat MCP authorization as a protocol baseline and add separate identity governance for workload attestation and delegated access.
Why agent gateways need identity context upstream
An agent gateway can span model traffic, MCP calls, APIs, and agent-to-agent communication, but its enforcement value depends on what identity context it receives. If the gateway only knows where to send traffic, it can route and observe but not decide authority correctly. Effective enforcement requires upstream decisions about the attested workload identity, the human or organisational identity represented in the interaction, the policy decision for the requested action, and the credential issued for access. Without that upstream context, the gateway becomes a traffic filter rather than an access boundary.
Practical implication: Build gateway enforcement on attested identity and policy outputs, not on the gateway’s own classification of the request.
Breaches seen in the wild
- LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Gateway terminology is obscuring a governance problem, not solving it. The industry is using one label for several different control points, but the decisive question is always authority, not routing. An LLM gateway, MCP gateway, or agent gateway can inspect and direct traffic, yet none of those labels tells you who is authorised to act. Practitioners should stop treating gateway category as a proxy for access control maturity.
Authority must be established before a request reaches the gateway decision point. This article makes the upstream dependency explicit: attested workload identity, represented-user context, policy evaluation, and credential issuance belong to the control plane, while the gateway enforces the result. That separation matters because enforcement without identity context becomes brittle as agentic systems spread across multiple tools and protocols. The practical conclusion is that gateway architecture and identity architecture have to be designed together.
Blended identity is the right lens for delegated agent actions. The article’s strongest contribution is the idea that an agent request can carry both the workload identity of the agent and the identity it represents. That is the right mental model for agentic access governance because it preserves accountability across delegation chains instead of collapsing everything into a shared service account. Practitioners should treat represented-user context as a first-class governance object, not an implementation detail.
Credential isolation is becoming a boundary condition for trustworthy agentic systems. When gateways keep destination credentials out of the agent and broker token exchange instead, they reduce the chance that authority becomes portable, reusable, or detached from policy. That is an identity control problem, not just a transport design choice. The broader implication is that enterprise AI architecture is moving toward governed delegation, where the control plane decides and the gateway enforces.
Semantic routing does not remove the need for policy discipline. Whether a gateway chooses a model, a tool, or an agent based on meaning, the decision still needs to be constrained by identity and access rules. Otherwise semantic convenience can become policy drift. Practitioners should view semantic gateways as decision accelerators, not as substitutes for explicit authorisation.
From our research library:
- Gartner predicts that AI systems will initiate 50% of all service requests by 2030, driven largely by agentic AI.
- Read next: AI Agent Identity Security Buyer's Guide
What this signals
Authority must be governed above the gateway layer. AI gateway consolidation is useful for traffic management, but it does not remove the need for a separate identity control plane that can attest workloads, evaluate delegated authority, and issue destination-scoped credentials. That separation is becoming a design requirement for any organisation exposing agents to production services.
Blended identity should become a standard design assumption for agentic access. Once an agent can act on behalf of a human or organisation, access decisions have to preserve both identities rather than collapsing them into a shared system token. That is the governance shift practitioners should plan for as agentic workflows move from pilots to operational systems.
For practitioners
- Separate routing from authority decisions Map which layer selects destinations, which layer evaluates policy, and which layer issues credentials so AI gateway labels do not blur control ownership.
- Require attested workload identity upstream Do not let a gateway enforce access unless the calling workload has been attested by an identity control plane before the request reaches the gateway.
- Preserve represented-user context Carry the human or organisational identity being represented through the request path so delegated actions remain accountable after routing and mediation.
- Broker destination-specific credentials Issue the credential required for the target service instead of allowing the agent or gateway to hold reusable long-lived credentials.
Key takeaways
- AI gateways can centralise routing and observability, but they do not by themselves establish delegated authority for agent actions.
- The article’s core distinction is between traffic enforcement at the gateway and identity decisions upstream in the control plane.
- Practitioners should design for attested workload identity, represented-user context, and destination-specific credentials before they rely on gateway enforcement.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST SP 800-53 Rev 5 and NIST Zero Trust (SP 800-207) set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-04 — Insecure Authentication | The article hinges on whether gateways can trust the caller’s identity before enforcing access. |
| NHI-05 — Overprivileged NHI | Delegated agent and service access must avoid broad reusable authority across tools. | |
| NHI-07 — Long-Lived Secrets | The article warns against agents or gateways holding reusable credentials instead of brokering them. | |
| Recommendation — Attest the calling workload before allowing gateway-enforced access decisions. Scope gateway-mediated access to the minimum destination-specific privilege required. Replace long-lived shared credentials with short-lived, destination-scoped issuance. | ||
| NIST SP 800-53 Rev 5 | IA-9 — Service and Organization User Identifier Authentication | Agent, workload, and API interactions depend on authenticating non-person identities to one another. |
| Recommendation — Apply IA-9 to authenticate service and workload identities before gateway authorisation. | ||
| MITRE ATT&CK | TA0006;TA0008 — Credential Access; Lateral Movement | Gateway and token-handling failures can expose credentials and widen downstream movement paths. |
| Recommendation — Map gateway credential handling to TA0006 and TA0008 when access control is delegated. | ||
| NIST Zero Trust (SP 800-207) | 6.1 — Policy Decision and Enforcement | The article separates the policy decision from the gateway enforcement point. |
| Recommendation — Keep policy decision logic upstream and use the gateway only for enforcement. | ||
Key terms
- AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
- Represented-User Context: Represented-user context is the human or organisational identity that an agent or workload acts on behalf of during a request. It preserves accountability across delegation chains and prevents access from collapsing into a shared technical identity with no business owner.
- Workload Identity: The identity assigned to a software workload, such as a containerised application, serverless function, or microservice, enabling it to authenticate to other services without storing static credentials.
- Blended Identity: Blended identity occurs when an autonomous system acts partly on behalf of a person and partly under its own machine authority. This creates split accountability because one actor may initiate the task while another identity performs the privileged action across different systems.
Deepen your knowledge
NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.
Published by the NHIMG editorial team on September 6, 2026.
Updated on October 7, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org