TL;DR: AI gateways matter less as cost optimisers than as control planes that centralise virtual keys, enforce guardrails once, and bring shadow MCP servers back under governance before credentials, prompts, and tool calls sprawl across agents and providers, according to JetStream Security. The security case is now architectural: control at the gateway, or inherit unmanaged AI access at scale.
At a glance
What this is: JetStream Security frames the AI gateway as a governance layer, not just a routing layer, because it can centralise credentials, apply policy once, and surface shadow MCP activity.
Why it matters: For IAM, PAM, NHI, and AI security teams, the implication is that control points must move closer to model access and tool delegation, where credential sprawl and agent misuse actually happen.
By the numbers:
- The average cost of an incident involving compromised credentials is $4.81 million.
- GitGuardian scanned nearly two billion public GitHub commits and found more than 28 million secrets pushed into public repos in a single year.
- Gravitee found that 47% of the roughly 3 million AI agents already deployed across surveyed organisations were not being actively monitored or secured.
- Wallarm counted 315 MCP-related vulnerabilities published in 2025 alone.
Context
AI gateways are becoming a governance layer because they concentrate the places where model access, routing, prompt inspection, and credential handling intersect. In practice, the control plane matters more than the model choice when multiple engineers, agents, and providers can otherwise accumulate keys and permissions outside normal oversight.
The article’s core claim is that the same gateway used for cost routing can also reduce credential sprawl and enforce policy across model traffic and MCP connections. That makes the identity problem visible: every agent, API key, and delegated tool call becomes easier to inventory when there is one control point rather than many scattered ones.
Key questions
Q: What breaks when AI gateways are not used to centralise model, agent, and tool traffic?
A: Without a central gateway, teams usually end up with fragmented controls, inconsistent logging, and weak policy enforcement across models, agents, and tools. That makes it harder to audit usage, contain misconfiguration, and coordinate incident response. In practice, the organisation loses a reliable view of what AI systems accessed, what they did, and whether they stayed within approved boundaries.
Q: Why do overprivileged API keys create more risk in agentic AI systems?
A: Because the key becomes the real enforcement point for everything the agent can do. If scope is too broad, one manipulated prompt can trigger actions across multiple systems, and the resulting damage depends on what the credential can reach rather than what the model intended.
Q: How should security teams govern shadow MCP servers discovered on managed devices?
A: Security teams should treat shadow MCP discovery as a governance and exposure problem, not just a tooling issue. The first priority is to identify where unmanaged servers are being configured, separate new or unknown instances from ones with managed alternatives, and review affected users and audit logs. That gives teams a practical path to reduce sprawl, prioritize migration, and limit hidden access paths.
Q: When is a gateway better than per-agent guardrails for AI security?
A: A gateway is better when many agents, teams, or providers share the same policy needs. Central enforcement reduces duplication and closes gaps that appear when each integration implements its own checks. Per-agent guardrails can help locally, but they do not solve the governance problem when access, filtering, and logging need to be applied consistently across the whole AI estate.
Technical breakdown
Why AI gateways become the control point for model credentials
An AI gateway sits between callers and model providers, so it can broker access instead of letting every engineer or agent hold a direct provider key. That changes the security model from distributed secrets to centrally governed virtual keys, scoped routing, and policy enforcement at one choke point. The important shift is not performance, but control locality: authorization, logging, and filtering happen before the request reaches the model or tool layer. This matters because the operational risk is not just model misuse, but unmanaged credential distribution across code, chat, and CI systems. See the LLM Provider API Key Security and LLMjacking Guide.
Practical implication: Treat the gateway as the enforcement point for NHI credential scope, rotation, and revocation.
How prompt and tool-call inspection changes AI governance
Prompt inspection and tool-call filtering are runtime controls that can block or sanitise risky instructions before an AI system acts. In agentic workflows, the model is not the only risk surface; the tool invocation is where data can move, actions can be triggered, and privilege can be abused. A central policy layer is valuable because it applies the same rules to every model and every integration rather than relying on each agent implementation to be safe by design. That is why gateway-based controls increasingly resemble a shared governance boundary rather than an optimisation feature. See the Agentic AI Security Policy Template.
Practical implication: Inspect every model request and tool call before execution, not after the fact.
Why MCP servers create a new identity and access problem
Model Context Protocol extends agent reach into databases, file systems, and internal APIs, which means the security question shifts to who or what is allowed to act through those connections. Shadow MCP servers are dangerous because they can be spun up quickly, remain outside inventory, and present as legitimate services to an agent. The result is not just an unmanaged integration, but an unauthenticated or weakly governed bridge between a model and production systems. From an identity perspective, the server, the agent, and the delegated credentials all need explicit governance. See the Shadow AI and AI Agent Discovery Guide.
Practical implication: Inventory MCP servers and bind each one to explicit authentication, authorisation, and ownership.
Threat narrative
Attacker objective: The attacker wants to turn trusted AI access paths into a durable foothold for model abuse, data access, or tool misuse.
- Entry occurs when engineers or agents acquire provider API keys, model access, or shadow MCP endpoints through scattered files, chats, or CI workflows.
- Credential exposure then creates standing opportunities for abuse because the same keys can be reused across multiple models, agents, and tools.
- Escalation follows when a compromised key or unvetted MCP server is used to reach internal systems, inspect prompts, or invoke tools beyond intended scope.
- Impact is credential sprawl, uncontrolled agent access, and potential data or system abuse through an unmanaged AI control plane.
Breaches seen in the wild
- LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI gateways are becoming the policy plane for agentic access, not just a routing layer. Once model traffic, prompt inspection, and tool calls converge in one place, the governance question moves from model choice to access control. That is the right architectural response to credential sprawl because direct provider access across many teams creates blind spots that no downstream audit can fully recover. The practitioner conclusion is straightforward: if the gateway is not your enforcement point, you probably do not have one.
Shadow MCP is the new unmanaged integration risk. The article is right to frame MCP servers as governance objects, because they can expose internal systems to agents without the same inventory, approval, or offboarding discipline used for conventional integrations. This is where shadow AI and AI agent discovery becomes operationally relevant: discovery has to include the protocol and the service, not just the model. The practitioner conclusion is to treat every MCP server as an identity-bearing workload, not a helper script.
Virtual keys only solve the problem if they are actually revocable and scoped. A gateway that simply hides provider credentials without constraining where they work still leaves an access-governance gap. The article’s stronger point is that the control value comes from centrally issued, rotated, and limited keys that are tied to policy and telemetry. The practitioner conclusion is to measure whether your AI control plane can revoke access faster than a leaked key can be abused.
The market is converging on control planes because governance has become the product requirement. When model vendors, endpoint vendors, and data platform vendors all point to a gateway pattern, the signal is not just architectural convenience. It is that enterprise buyers now expect inspection, policy, and inventory at the AI boundary rather than in scattered point solutions. The practitioner conclusion is to evaluate AI platforms by where they enforce control, not by how many models they can route.
What this signals
AI governance is moving from model selection to control-plane design. The decisive issue is no longer which model a team chooses, but where policy lives and who can change it. As AI workloads spread, security teams need one place to govern credentials, prompt handling, and tool delegation before those decisions become embedded in every agent and integration.
Shadow MCP turns agent governance into an inventory problem. If the protocol layer is not discovered, classified, and owned, the organisation cannot reliably say which systems an agent can touch. That makes AI governance depend on the same discipline used for service accounts and other non-human access paths, with the difference that the acting entity may change behaviour at runtime.
For practitioners
- Centralise AI credential issuance Move provider keys out of engineer laptops, .env files, chat threads, and CI jobs, and issue scoped virtual keys from one governed control point.
- Enforce prompt and tool-call policy at the gateway Inspect prompts, tool requests, and outbound responses before they reach a model or external system, and block or sanitise unsafe content consistently across providers.
- Inventory every MCP server Record ownership, authentication method, data reach, and business purpose for each MCP endpoint so shadow integrations do not remain outside review.
- Rotate and revoke AI keys on a fixed cadence Tie rotation to measurable exposure windows, and make revocation available from the same place that issues access so leaked secrets can be cut off quickly.
Key takeaways
- AI gateways are now being positioned as governance controls because they can centralise access, policy, and inspection across model traffic and tool calls.
- The article ties AI risk to real exposure patterns, including compromised credentials, leaked secrets, and unmanaged MCP endpoints.
- The practical shift for teams is to treat the AI boundary as an identity and access control point, not just a performance or cost optimisation layer.
Related NHI Mgmt Group resources: Shadow AI and AI Agent Discovery Guide · LLM Provider API Key Security and LLMjacking Guide · Agentic AI Security Policy Template
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10, OWASP Agentic AI Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | The article centres on exposed provider keys and other AI credentials leaking into unmanaged locations. |
| NHI-05 — Overprivileged NHI | Gateway design is about reducing excessive access for agents, integrations, and provider keys. | |
| NHI-07 — Long-Lived Secrets | The article argues for virtual, revocable keys instead of persistent provider secrets. | |
| Recommendation — Scan for leaked AI credentials and revoke exposed keys before they can be reused. Scope AI credentials to the minimum model, tool, and environment access required. Replace durable AI keys with short-lived, revocable credentials issued from a control plane. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Agent tool use and delegated access are central to the article's governance concerns. |
| Recommendation — Apply identity checks and privilege boundaries before agents can invoke tools or reach data. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article is fundamentally about establishing a governed AI control plane. |
| Recommendation — Define ownership, policy, and accountability for AI access decisions at the gateway. | ||
| MITRE ATT&CK | TA0006; TA0008 — Credential Access; Lateral Movement | Leaked keys and exposed MCP servers create paths for credential abuse and movement into internal systems. |
| Recommendation — Map exposed AI credentials and MCP footholds to credential access and lateral movement detections. | ||
Key terms
- AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
- Shadow MCP: Shadow MCP is an unauthorized or ungoverned use of the Model Context Protocol inside an organization. It appears when AI agents, scripts, or tools connect to data or actions through MCP servers without security review, inventory, policy control, or monitoring. This creates hidden access paths, data exposure, and weak accountability.
- Virtual key: A scoped credential issued for a specific team, environment, or use case rather than a broad shared secret. It reduces blast radius by making model access easier to attribute, limit, and revoke across different parts of the organisation.
- Tool Call Inspection: Tool call inspection is the process of examining data returned by an external system before an AI agent receives it. In practice, this means checking payloads for sensitive fields, applying policy-based redaction, and recording evidence for audit and response. It is a key control for safe AI use with enterprise applications.
Deepen your knowledge
NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management in a way that helps teams connect identity controls to operational risk. It is a useful baseline for practitioners who need to govern AI access without losing sight of lifecycle and privilege.
Published by the NHIMG editorial team on October 22, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org