TL;DR: As organizations move from LLM experiments to production AI, Kong says fragmented model, MCP, and API integrations create governance, observability, and security gaps that edge inspection and gateway control are meant to close. The core issue is not model choice but control-plane sprawl, where policy, routing, and data protection are no longer centrally enforceable.
At a glance
What this is: This is Kong’s architecture piece on securing production AI infrastructure, arguing that AI gateway, MCP, and edge controls are needed because fragmented model access and tool integration create governance and data-protection gaps.
Why it matters: IAM, NHI, and platform teams should treat AI connectivity as a control-plane problem, because authentication, routing, observability, and policy enforcement now need to span models, tools, and APIs together.
Context
Production AI systems now depend on multiple LLM providers, MCP servers, and REST APIs rather than a single model endpoint. That shifts the governance problem from model selection to control-plane consistency, because authentication, routing, logging, and data handling must remain coherent across all paths.
The article frames the gap as an AI infrastructure issue, not a model-quality issue. Without a central orchestration layer, teams lose visibility into how requests move across gateways, tools, and providers, which makes policy enforcement and exfiltration control harder to sustain.
This is best understood as an AI gateway and edge-security pattern for agentic systems. The architecture described is typical of the direction the market is taking, where operational control matters more than any single model choice.
Key questions
Q: How should security teams govern AI gateways in production environments?
A: Security teams should govern AI gateways like shared control planes, not convenience proxies. That means tying every key, token, and routing policy to an owner, enforcing least privilege for configuration changes, and making logging, retention, and fallback behaviour auditable. The gateway should sit inside IAM, secrets, and incident response processes, not beside them.
Q: Why do fragmented LLM and MCP integrations increase security risk?
A: Because each integration can introduce a separate trust boundary, teams lose the ability to enforce one consistent rule set for routing, observability, and data handling. The result is control-plane sprawl, where sensitive traffic moves through multiple services without a single source of truth for policy.
Q: What are the signs that an AI gateway is not operating effectively?
A: Common warning signs include inconsistent API behavior across providers, weak audit trails, poor visibility into who called which model, and rising costs from duplicate or repeated prompts. If teams cannot track latency, usage, and guardrail triggers, the gateway is failing its core job. A reliable gateway should provide clear observability, stable routing, and auditable request-response records.
Q: What should teams do when MCP tools are added without access scoping?
A: Pause deployment until each tool has an explicit business purpose, a minimum permission set, and a named approver. Broad or default access should be treated as a governance defect, not a convenience. Without scoping, the system can invoke functions the business never intended, which is exactly how delegated access becomes abuse.
Technical breakdown
Why AI gateway layers are emerging around LLM and MCP traffic
Modern AI applications increasingly mix model providers, tool servers, and standard APIs in one request path. That creates inconsistent policy surfaces, because each integration may authenticate differently, expose different metadata, and log different events. A gateway normalizes those paths so the platform can apply one set of controls for routing, rate limiting, observability, and authorization. In practice, the gateway becomes the control point where model selection, tool access, and request inspection can be coordinated instead of managed in separate silos.
Practical implication: Treat the AI gateway as the enforcement point for model, tool, and API access rather than as a traffic relay.
How MCP changes the control problem for AI systems
MCP turns external tools into callable capabilities for AI systems, which makes tool access a governance issue as much as an integration issue. If MCP servers expose broad tool inventories, the system must distinguish which tools are available, which identities may call them, and what policies apply to each request. That is why proxying, authN, authZ, and policy enforcement matter. The risk is not just tool sprawl, but unmanaged delegation of action paths that can reach sensitive data or privileged backend services.
Practical implication: Map each MCP tool to an explicit authorization path and avoid leaving tool inventories broadly callable by default.
Why edge inspection matters for prompt injection and data exfiltration
Edge inspection is useful because it can evaluate requests and responses before they reach models or leave the platform. Prompt injection and exfiltration are not model bugs alone; they are path-control problems, where malicious instructions or sensitive outputs move through the same channels as legitimate traffic. Filtering at the edge helps separate benign prompts from unsafe intent and gives the platform a chance to block harmful content before it propagates downstream or returns to the user.
Practical implication: Place content inspection and egress filtering close to the request boundary so unsafe prompts and sensitive outputs are intercepted early.
Threat narrative
Attacker objective: The objective is to manipulate AI workflows so the attacker can extract sensitive data or steer agent behaviour through trusted infrastructure paths.
- Entry occurs through normal AI request paths when users or agents submit prompts, model queries, or MCP tool calls into a fragmented stack.
- Escalation follows when the platform lacks a single policy plane, allowing tool calls, routing decisions, and provider access to drift beyond intended governance.
- Impact appears as prompt injection, data exfiltration, and loss of visibility into which model or tool handled the sensitive interaction.
Breaches seen in the wild
- LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
- ShadowRay 2024: Attackers took over internet-exposed Ray AI clusters via the disputed CVE-2023-48022, exposing cloud keys, AI API tokens and GPU compute.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI control-plane sprawl is now the core governance problem for production AI: once organizations combine multiple model providers, MCP servers, and API backends, security is no longer enforced by any single control point. Policy, logging, and routing become distributed across components that were not designed to behave like one identity plane. The practitioner conclusion is that AI governance now depends on making the control plane explicit.
MCP exposes the difference between integration and authorization: a tool being reachable does not mean it should be callable by every agent or user context. This is a classic NHI lesson applied to AI connectivity, where broad exposure becomes privilege leakage if tool access is not scoped per request and per identity. The practitioner conclusion is to treat tool inventories as governed access surfaces, not integration conveniences.
Prompt injection and data exfiltration are path-control failures, not just model-safety issues: the risk emerges when untrusted instructions and sensitive outputs can traverse the same request path without inspection. Edge controls matter because they give teams a boundary at which intent, content, and egress can be evaluated before downstream systems act. The practitioner conclusion is that AI security must be enforced at the traffic layer as much as at the model layer.
Semantic routing creates a new kind of identity blast radius: when a gateway can steer requests by cost, latency, or context, the decision authority around model selection becomes part of the security posture. That makes observability and authorization inseparable, because routing choices can change where data lands and which provider handles the request. The practitioner conclusion is that routing logic must be governed like privileged infrastructure logic, not treated as a mere performance optimization.
Runtime AI governance assumes a stable request path, and that assumption is already breaking: the model stack was designed for fixed provider relationships and known tool paths. That assumption fails when agent frameworks dynamically combine LLMs, MCP servers, and backend APIs at runtime. The implication is not just tighter controls, but a redesign of how identity, policy, and telemetry are anchored in the AI workflow.
From our research library:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
- Read next: AI Agent Identity Security Buyer's Guide
What this signals
AI control-plane sprawl: once AI systems mix models, MCP tools, and APIs, governance has to move from scattered integration points to a single policy layer. That shift is already visible in the architecture pattern Kong describes, where the control point becomes the only place to make routing, authorization, and logging consistent.
The next programme question is whether your team can prove that prompt inspection, tool authorization, and model routing all happen under one operating model. If not, you are relying on security properties that disappear as soon as the agent stack expands.
AI-related credential leaks surged 81.5% year-over-year in 2025, according to the State of Secrets Sprawl 2026, which is a reminder that adjacent AI infrastructure often weakens faster than the models themselves.
For practitioners
- Define a single AI control plane Standardise how authentication, routing, logging, and policy enforcement are applied across LLM providers, MCP servers, and REST APIs so governance does not fragment by integration.
- Inventory MCP tools and access paths List every tool exposed through MCP, the identities that can call it, and the policies that should restrict each call path before production rollout.
- Enforce edge inspection for prompts and outputs Apply request and response filtering at the boundary to catch prompt injection, toxic output, and data exfiltration before they reach internal services or users.
- Separate provider choice from authorization Make model routing decisions independent from entitlement decisions so a new provider or agent framework does not inherit broader access than intended.
Key takeaways
- Production AI becomes harder to govern once model providers, MCP tools, and APIs are stitched together without a common control plane.
- The most useful security boundary is the one that can inspect, route, and log traffic before requests reach models or tools.
- Teams should focus on explicit authorization, edge filtering, and unified observability rather than treating model choice as the primary control problem.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, CSA MAESTRO, OWASP Non-Human Identity Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI02 — Tool Misuse | The article focuses on controlling how agents use external tools and MCP servers. |
| ASI03 — Identity & Privilege Abuse | Gateway policy gaps can let agents inherit or exceed intended privileges across integrations. | |
| Recommendation — Restrict agent tool access to explicitly authorised actions and inspect every tool invocation. Enforce identity-bound privilege checks before an agent can route, call, or chain tools. | ||
| CSA MAESTRO | Agentic AI threat modeling | The post is about threat boundaries in a multi-layer agentic AI architecture. |
| Recommendation — Model trust boundaries across LLM, tool, and edge layers before production rollout. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article is fundamentally about governance of AI routing, logging, and access control. |
| Recommendation — Establish accountable governance for AI routing, access, and observability across the stack. | ||
| OWASP Non-Human Identity Top 10 | NHI-04 — Insecure Authentication | MCP and gateway traffic depend on strong authentication between services and tools. |
| Recommendation — Harden authentication for every model and tool endpoint before expanding AI access. | ||
| MITRE ATT&CK | TA0006;TA0010 — Credential Access; Exfiltration | The article explicitly discusses prompt injection and data exfiltration risks. |
| Recommendation — Map AI traffic abuse to credential access and exfiltration tactics in detection coverage. | ||
Key terms
- AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
- Model Context Protocol: Model Context Protocol is an open protocol that lets AI agents connect to tools and data sources. It expands what an agent can reach, so governance has to cover not only the model and its prompts, but also every system that can receive or return agent-driven data.
- Control-plane sprawl: Control-plane sprawl is the condition where identity, device, privileged access, and provisioning are managed in separate systems with weak policy coordination. It usually creates duplicated administration, inconsistent enforcement, and slower offboarding because no single workflow owns the whole lifecycle.
- Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
Deepen your knowledge
NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.
Published by the NHIMG editorial team on June 23, 2026.
Updated on October 11, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org