AI gateways vs MCP gateways: where each control layer fits

At a glance

What this is: This analysis separates AI gateways from MCP gateways and finds that each controls only part of the agent path, leaving the full session and non-MCP actions outside both layers.

Why it matters: IAM, NHI, and agentic AI teams need to treat gateways as partial controls, because identity-aware runtime governance and session enforcement still require another layer.

Context

AI gateways and MCP gateways are different control points, not interchangeable labels. The article argues that one manages model inference traffic while the other manages tool traffic, but neither fully governs what an agent does across the session.

That distinction matters because enterprises are already trying to use gateways as security boundaries for agentic systems. When traffic can bypass the gateway through native commands, local transports, or vendor-managed agent platforms, the control plane no longer matches the real execution surface.

The governance problem is broader than protocol routing. For identity programmes, the issue is whether access, attribution, and enforcement still hold when the agent can act through multiple channels and outside a single proxy layer.

Key questions

Q: Where do API gateways and directories fail to govern AI agents in practice?

A: They fail at execution time. Gateways can inspect requests and directories can authenticate a principal, but neither owns what the agent does after access is granted. If the control must block, attribute, or meter an action while it is happening, governance has to move into the workload path rather than remain outside it.

Q: Why does session context matter more than per-request filtering for agents?

A: Because an action can only be judged properly when you know what happened earlier in the same session. A tool call that looks safe on its own may become risky after the agent has already handled sensitive data or received a prompt to exfiltrate it. Per-request filters miss that sequence, while session context preserves it.

Q: What do teams get wrong about MCP security when they rely on basic gateways alone?

A: Teams often confuse routing with security. A basic gateway can move messages, but it does not solve malformed parameters, missing validation, weak audit trails, or tool name collisions. In practice, that means the agent can still reach unsafe execution paths unless the system validates inputs, namespaces tools, and records every action centrally.

Q: How should organisations govern identity when agent creators and users are different?

A: They should treat the creator’s credentials, the current user, and the agent’s effective permissions as separate governance questions. If a shared agent runs under the maker’s identity, access checks that only confirm login state will miss who really has authority at runtime. Identity governance has to follow the effective permission chain, not just the person who built the agent.

Technical breakdown

AI gateway vs MCP gateway: different traffic, different control points

An AI gateway sits between an application or agent and the LLM provider, so it sees prompts, responses, routing decisions, and usage data for model inference. An MCP gateway sits between the agent and MCP servers, so it sees tool requests and tool responses. The architectural difference matters because one protects the model path and the other governs tool access, but neither is a full session security layer. If the agent executes native bash commands, writes files, or makes direct HTTP calls, those actions fall outside both gateway models. The control boundary is therefore transport-specific, not agent-wide.

Practical implication: Treat the gateway choice as a control boundary decision, not a complete security design.

Why gateway visibility is not the same as enforcement

Visibility does not equal governance. AI gateways can observe tool calls because they sit on the inference path, but they were not designed to enforce per-tool authorisation, tool registries, or session-level policy decisions. MCP gateways can authenticate users, scope tool access, and log calls, but they only govern what is explicitly routed through them. That leaves a large enforcement gap when developers configure local agents, use stdio transports, or operate prebuilt assistants that bypass customer-controlled routing. The result is a partial record of activity without reliable control over all execution paths.

Practical implication: Use gateway logs as evidence, but do not mistake them for complete enforcement coverage.

Why session context is the missing security layer

Security decisions in agentic systems depend on the sequence of actions, not just individual requests. A tool call can be harmless in isolation and risky later in the same session, especially when sensitive data, external actions, and model-driven decisions combine. Gateways generally see one slice of that interaction. They do not keep full session context, the user’s original intent, or the progression of prior actions that changes whether a later tool call is acceptable. That is why static rules like blocking all email or all deletion are too blunt, while nuanced behavioural enforcement requires a deeper runtime layer.

Practical implication: Design policy around session-wide behaviour, not isolated request inspection.

Breaches seen in the wild

• LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
• 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Gateway control is only transport control: AI gateways and MCP gateways divide the agent path into model traffic and tool traffic, but that partition does not equal complete governance. The article shows that native commands, direct API calls, and vendor-managed agent platforms sit outside the proxy boundary. That means the industry is still solving slices of the problem, not the whole agent security model. Practitioners should stop treating gateway coverage as synonymous with runtime control.

Session context is the real governance gap: The security assumption behind gateway-based control is that meaningful risk can be judged at the request level. That assumption fails when an agent’s later action becomes risky only because of earlier prompts, tool results, or data access in the same session. The implication is that agent governance has to move from per-request filtering to session-aware enforcement. Practitioners should re-evaluate any design that assumes isolated traffic inspection can secure multi-step agent behaviour.

Identity-aware enforcement cannot stop at authentication: The article’s maker’s token example shows that who created an agent and who uses it later can diverge, so static credential checks are not enough. Gateway integration with identity providers is useful, but the harder problem is resolving effective permissions at runtime across the full agent session. That is where identity governance and behavioural security converge. Practitioners should treat effective authority, not login success, as the control point.

Converged gateway products will not erase the control gap: The market is moving toward bundled AI gateway and MCP gateway features, but convergence at the infrastructure layer does not eliminate the missing security layer. Routing, authentication, and logging may merge, while behavioural detection and full-session enforcement still remain separate problems. That means platform decisions should focus on whether a control can actually govern agent action, not whether it sits under one product label. Practitioners should evaluate architecture by coverage, not by category names.

From our research library:

• 40 percent of financial and software companies have already deployed agentic AI systems, and deployments are expected to double by 2028.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

Session governance is becoming the deciding control plane: Gateway coverage will keep improving, but the core problem is not whether traffic can be routed. The problem is whether a security programme can reason over the full interaction, including model output, tool use, native execution, and identity context. That is where agentic AI governance shifts from infrastructure management to runtime decision control.

AI gateway and MCP gateway are partial controls, not competing end states: The market will keep converging products, but convergence does not remove the need to distinguish transport visibility from behavioural enforcement. Security teams should expect the next gap to show up wherever an agent can act outside the routed path, especially in local execution and prebuilt SaaS environments.

For practitioners

• Define the real control boundary Map which agent actions are actually routed through an AI gateway, an MCP gateway, or neither, then document the blind spots created by native commands, local transports, and vendor-managed platforms.
• Separate authentication from runtime authority Review whether your agent governance model only checks who logged in, or whether it also resolves effective permissions when the maker of an agent differs from the current user.
• Instrument the session, not just the request Require telemetry that correlates prompts, tool calls, model responses, and downstream actions so policy can evaluate an agent’s behaviour across the full interaction.
• Track shadow MCP usage outside gateways Use endpoint telemetry, EDR, and network monitoring to find MCP servers and agent activity that never reaches the gateway and therefore never appears in gateway logs.

Key takeaways

• AI gateways and MCP gateways each govern part of the agent stack, but neither provides complete runtime control across all agent actions.
• The biggest gap is not naming or terminology. It is the absence of session-wide enforcement that can follow identity, intent, and tool use across a multi-step interaction.
• Security teams should evaluate gateway coverage against the full execution path, then add a separate control layer for behavioural detection and identity-aware governance.

Key terms

• AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
• MCP Gateway: The control layer that relays assistant intent to tools and data sources through the Model Context Protocol. In practice, it becomes a policy boundary, not just a transport layer. If it trusts model output too early, it can turn unverified reasoning into real-world execution or disclosure.
• Session context: The surrounding authentication and access conditions attached to a login, such as MFA status, device trust, IP reputation, and prior behaviour. It matters because identity risk is rarely decided by one event alone; context shows whether the event fits the account’s normal pattern.
• Maker's Token: A permission pattern in which an agent or application continues to operate using the credentials of the person who created it, even when another user is driving it later. This can create hidden authority transfer and make current-user checks misleading unless effective permissions are resolved at runtime.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.