TL;DR: AI gateways centralise prompt inspection, routing and spend controls, but they do not govern how an agent reaches internal databases, APIs or MCP servers, according to Twingate. That leaves a separate network access layer to enforce least privilege, identity and blast-radius limits.
At a glance
What this is: NHI Mgmt Group's analysis, drawing on Twingate's blog post: AI gateways do not secure the path from an agent into internal resources, so identity-aware network access becomes the missing control layer.
Why it matters: IAM, PAM and NHI teams need to treat agent-to-resource connectivity as a distinct control plane because prompt filtering does not constrain what the agent can reach inside the enterprise.
Context
AI gateways are often described as the control point for model traffic, but that framing stops at the prompt boundary. The real governance gap appears when an agent needs to reach a private database, internal API or MCP server inside the enterprise network.
That gap is a resource-access problem, not a model-safety problem. If the connectivity path still depends on broad VPNs, exposed ports or static allowlists, the organisation has moved the risk rather than reducing it.
Key questions
Q: How do AI gateways differ from zero trust access control?
A: AI gateways govern model traffic, including prompt logging, token accounting, routing, and content filtering. Zero trust access control governs what an agent can reach after the model responds, such as internal APIs, databases, admin tools, and other private resources. The two layers solve different problems and are both needed when agents can act on their outputs.
Q: How should teams limit an AI agent's access to internal systems?
A: Teams should grant the agent access only to the specific resource it needs, not to an entire subnet or VPC. Resource-scoped policy, identity-based enforcement and outbound-only connectivity reduce the blast radius if the agent or its host is compromised.
Q: Why is IP allowlisting weak for agent access?
A: IP allowlisting is weak for agent access because an agent can run from changing cloud addresses, shared infrastructure or delegated execution paths, so the source IP does not reliably identify the caller. Identity-based access ties each agent, service account or workload to authenticated credentials, then applies scoped permissions, token validation and revocation.
Q: What is the safest way to expose an internal API to an AI workflow?
A: The safest way is to front the internal API with an identity-aware policy layer that authenticates the calling workload or agent, validates its token or certificate, and authorises only the exact methods and data scopes required. Combine that with short-lived credentials, mTLS and per-route access rules rather than flat network exposure.
Technical breakdown
Why prompt gateways do not govern resource access
An AI gateway sits between the application and the model provider, where it can inspect prompts, apply policy, route requests and record usage. It does not sit on the path to private infrastructure, so it cannot decide whether an agent should reach an internal API, database or MCP server. That separation matters because model traffic and resource traffic have different trust boundaries, different audit requirements and different blast-radius profiles. See the AI Gateway and LLM Gateway Guide.
Practical implication: Treat model governance and resource access as separate control problems, with separate enforcement points.
How ZTNA constrains an agent's network reach
Zero Trust Network Access works by binding access to identity and specific resources rather than to a subnet or perimeter. A connector inside the private network brokers outbound-only access, while policy is enforced on both ends of the tunnel. The result is resource-level reachability, not network-wide presence, which is the right grain when an agent only needs one hostname or API.
Practical implication: Scope agent connectivity to named resources and deny network-wide access by default.
Why agents create a different access profile from users
A human user usually arrives with device posture, interactive authentication and a slower-paced review cycle. An agent can generate large volumes of requests, operate continuously and inherit access from the host or service account that runs it. That changes the control objective from session protection to continuously bounded machine access, with identity, not IP address, as the meaningful security primitive. See the AI Security Architecture Reference Guide.
Practical implication: Bind agent access to an explicit non-human identity and avoid host-wide or subnet-wide entitlement.
Threat narrative
Attacker objective: The attacker uses the agent's standing connectivity to reach internal systems, move from a prompt-facing workload into private resources and expand access beyond the intended tool path.
- Entry occurs when an AI workload is given broad network reach, such as a public API key, VPN membership or host-level trust that exposes internal resources.
- Escalation follows when the agent or its host is compromised, because the attacker inherits whatever resource scope the agent was granted.
- Impact is reached when that inherited scope allows database reads, API actions or tool use across private infrastructure with little or no resource-specific containment.
Breaches seen in the wild
- LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
- LiteLLM supply chain attack 2026: Stolen GitHub CLI OAuth tokens let an attacker push malicious LiteLLM 1.107.0 to PyPI and Docker, stealing cloud, AI tool and LLM keys.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI gateway controls stop at the model boundary, so they cannot be the primary control for internal resource access. That boundary leaves a separate trust problem behind the agent, where private APIs, databases and MCP servers live. The implication is that identity governance for AI workloads must distinguish prompt safety from resource authorisation.
Standing network reach is the wrong default for agentic workloads. A persistent VPN or broad allowlist turns one compromised process into enterprise-wide reach. The stronger concept here is resource-scoped connectivity: the agent should only ever see the specific service it has been authorised to touch, not the enclosing network.
Agent identity should be enforced at the network layer, not inferred from the host. Host-based trust collapses when many processes, tools and dependencies share the same runtime. That means existing assumptions about device trust and user-style access reviews do not survive machine-paced access patterns, especially when a single agent can issue thousands of calls.
Zero Trust Network Access gives AI infrastructure a missing enforcement plane, but only if it is applied per-resource. This is where NHI governance and network governance converge: the same non-human principal that consumes model output can also become the gate to internal systems. Practitioners should treat that as a governance boundary, not a convenience layer.
What this signals
The access model for AI workloads is shifting from prompt governance to resource governance. Once an agent needs to reach internal systems, the deciding question becomes who can authorise that path and how tightly the path is constrained.
Resource-scoped connectivity: this is the governance pattern that matters when an agent can touch private infrastructure. It forces teams to stop thinking in terms of network membership and start thinking in terms of named services, named identities and revocable reach.
ZTNA becomes more relevant as agents move from chat-style interactions into operational tasks that affect data, workflows and infrastructure. The programme implication is simple: if the workload can act, the access path must be identity-bound and narrow enough to survive compromise.
For practitioners
- Define a separate control plane for agent-to-resource traffic Map every private database, API and tool endpoint an agent can reach, then assign explicit policy to each resource instead of inheriting network-wide access.
- Replace broad subnet trust with resource-scoped access Remove default VPN or host-wide access paths for AI workloads and constrain each agent to the exact hostname or service it needs.
- Bind agent activity to a non-human identity Use a service account or equivalent identity for the agent so logs, policy and revocation work at the identity layer rather than the IP layer. See the Agentic AI in IAM Operations Guide.
- Audit the blast radius of existing connectivity Review whether a compromised agent would inherit access to an entire subnet, a single resource or only a narrowly defined service path.
- Separate model controls from infrastructure controls Keep prompt inspection, routing and cost controls in the AI gateway, but enforce infrastructure reachability in the network layer where the resource actually lives.
Key takeaways
- AI gateway controls do not extend to the network path an agent uses to reach private infrastructure, so access governance must continue beyond prompt security.
- The operational risk is broad inherited reach, especially when agents run behind VPNs, bastions or static allowlists that were never designed for machine-paced access.
- Resource-level ZTNA, explicit non-human identity and narrow service scoping are the controls that change the blast radius when an agent is compromised.
Related NHI Mgmt Group resources: AI Gateway and LLM Gateway Guide · AI Security Architecture Reference Guide · Agentic AI in IAM Operations Guide · Shadow AI and AI Agent Discovery Guide
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 and NIST Zero Trust (SP 800-207) set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Agent identity and privilege boundaries are central when workloads reach private infrastructure. |
| Recommendation — Constrain agent privileges to the exact resources and actions the runtime is allowed to use. | ||
| OWASP Non-Human Identity Top 10 | NHI-05 — Overprivileged NHI | Broad network reach gives an AI workload more privilege than its task requires. |
| NHI-04 — Insecure Authentication | Agent access depends on how the workload is authenticated to private services. | |
| Recommendation — Reduce standing access and scope each non-human identity to named resources only. Use strong, identity-bound authentication for workloads instead of shared secrets or host trust. | ||
| NIST SP 800-53 Rev 5 | IA-9 — Identification and Authentication (Non-Organizational Users) | AI agents and service identities require explicit authentication before reaching internal services. |
| Recommendation — Authenticate non-human callers with identity-specific credentials before granting resource access. | ||
| NIST Zero Trust (SP 800-207) | Policy enforcement and resource access — Policy enforcement and resource access | The article's core control question is whether resource access is enforced outside the model path. |
| Recommendation — Enforce per-resource policy at the network boundary and deny broad subnet access. | ||
Key terms
- AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
- Zero-Trust Network Access: Zero-Trust Network Access is a way to give users secure access to specific applications without placing them on the internal network. Access is granted only after identity, device posture, and policy checks, then limited to the requested resource. ZTNA reduces lateral movement by replacing broad network trust with per-session authorization.
- Non-Human Identity (NHI): A digital identity assigned to a non-human entity such as a software application, service account, API key, bot, machine, or AI agent that enables it to authenticate and interact with systems without direct human involvement. NHIs now outnumber human identities in most enterprises by 25 to 50 times.
- MCP Server: An MCP server is a tool endpoint that connects an AI agent to external systems and data sources through Model Context Protocol. Because it extends what the agent can reach, it becomes part of the identity and access surface and must be reviewed like any other privileged connector.
Deepen your knowledge
NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM or security programme, it is worth exploring.
Published by the NHIMG editorial team on November 6, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org