What security teams actually need from an AI gateway

At a glance

What this is: This analysis argues that an AI gateway only matters if it can see, govern, and audit AI traffic across apps, desktop tools, web interfaces, and embedded SaaS features.

Why it matters: It matters because IAM, IGA, PAM, and security teams need a control point that distinguishes sanctioned AI use from shadow AI and can bind actions to accountable identities.

By the numbers:

• 20% of breached organizations were compromised through shadow AI, adding roughly $670,000 to the average breach cost.
• 97% of those organizations lacked proper access controls.
• 63% had no AI governance policy at all.

👉 Read TruFoundry's full analysis of what security teams need from an AI gateway

Context

AI gateway governance is about seeing, controlling, and auditing AI traffic before it becomes an untracked path into data and internal tools. In this article, TruFoundry frames the problem as one of visibility and enforcement across multiple AI entry points, which is exactly where traditional IAM and CASB-style thinking starts to fragment.

For identity teams, the important question is not whether AI exists in the environment. It is whether the organisation can bind AI usage to a accountable identity, scope access to specific tools or data, and preserve an audit trail when the actor is a person, a service account, or a non-human identity acting on behalf of either.

Key questions

Q: How should security teams govern AI in the security stack?

A: Security teams should treat AI as a governed decision aid, not an autonomous authority. Define where it can assist detection, prioritisation, and enrichment, then require human or policy approval for privileged actions and access decisions. The key control is traceability, so every AI-supported recommendation can be reviewed, challenged, and overridden.

Q: Why does residual risk matter for NHI and IAM programmes?

A: Residual risk matters because identity controls often reduce, but do not eliminate, the reach of service accounts, tokens, OAuth grants, and privileged access. Those identities can remain active after approvals change, which leaves a governed-looking environment with hidden runtime exposure. The programme goal is to make that remainder visible and defensible.

Q: What do security teams get wrong about AI gateway deployment?

A: They assume visibility alone equals control. In reality, logging and routing do not fix overprivilege, weak delegation, or unscoped tool access. A gateway that cannot tie a request to a revocable identity and a bounded action path only gives the appearance of governance.

Q: How do organisations know whether an AI gateway is actually working?

A: Look for three signals at once: AI traffic is inventoried, identity is preserved through the call chain, and audit records are usable in incident response or compliance review. If any one of those is missing, the gateway is serving as a transport layer, not a governance control.

Technical breakdown

AI gateway governance across four traffic paths

An AI gateway is not just a reverse proxy for model calls. In practice, it sits between users or applications and model endpoints, then applies routing, credential handling, logging, and policy enforcement. The key technical distinction is whether the gateway can control traffic from developer-built apps, desktop clients, consumer web apps, and embedded SaaS AI features. Those four paths behave differently, so a single control model rarely covers them all. Where the gateway cannot intercept traffic, governance shifts to device, endpoint, or vendor-admin controls.

Practical implication: map each AI entry path to the control that can actually enforce policy, rather than assuming one gateway reaches everything.

Scoped identity and token exchange for AI and MCP access

AI gateways become governance infrastructure when they stop treating every call as anonymous traffic and instead preserve identity through the chain. That means using scoped credentials for applications, separating an agent's identity from its registration, and exchanging tokens at each hop so the acting subject stays visible. This is especially important for MCP because tool calls can write, send, or modify real systems, which makes coarse server-level permissions too blunt. The mechanism is identity propagation plus least privilege at the tool boundary.

Practical implication: require per-application or per-agent identity, and do not allow shared tokens to mask who actually invoked the tool.

Runtime guardrails and audit trails for AI requests

Once traffic is visible, the gateway's next job is payload inspection and evidence retention. Guardrails can detect PII, secrets, prompt injection, and policy violations in requests and responses, while logging preserves the who, what, and when for SIEM export or incident reconstruction. The technical point is that auditability only works if identity, prompt content, model response, and cost all live in the same event stream. Without that correlation, security teams can see volume but not accountability.

Practical implication: verify that logging, redaction, and policy decisions remain correlated in a single audit record.

NHI Mgmt Group analysis

AI gateways are becoming the governance layer for shadow AI, not just the transport layer for model traffic. The article is strongest when it treats AI usage as an identity problem rather than a routing problem. Shadow AI becomes a control failure when security can neither inventory the actor nor bind its activity to policy. That means the gateway's value is measured by governance completeness, not latency or routing convenience.

The identity model behind AI usage has already fragmented across human, NHI, and agentic behaviour. Employees use web apps, desktop tools, and embedded SaaS features, while internal agents and scripts call models and tools through scoped credentials. The practical consequence is that IAM teams need one governance model that can distinguish human intent, non-human execution, and agent delegation without collapsing all three into a generic access layer.

Shadow AI is a standing privilege problem when organisations cannot tell which identities are allowed to reach AI systems. The article's emphasis on SCIM, JIT, and token exchange points to a broader truth: uncontrolled AI access behaves like overprivileged NHI sprawl. That is why the right question is not whether AI is fast enough to use, but whether every AI path is revocable, attributable, and reviewable.

Runtime policy only works when the gateway can inspect both the request and the action path. PII redaction, prompt-injection detection, and secrets filtering matter, but they do not solve governance if the resulting action still inherits excessive trust downstream. Practitioners should treat the gateway as an enforcement boundary for data, identity, and tool scope together, because one without the others creates a false sense of control.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, according to AI Agents: The New Attack Surface report.
• 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• For related governance context, see OWASP Agentic AI Top 10 for the control failures that most often drive tool misuse and scope drift.

What this signals

Shadow AI will keep expanding unless organisations treat AI access as an identity lifecycle problem. The control question is no longer whether an AI gateway exists, but whether every AI path can be provisioned, reviewed, and revoked with the same discipline used for other privileged identities. For a broader control map, practitioners should align this work with the OWASP Non-Human Identity Top 10 and the NIST AI Risk Management Framework.

Identity propagation will become the differentiator between real governance and cosmetic logging. If an organisation cannot preserve the acting identity across model calls, MCP tool access, and downstream API execution, it will not be able to answer basic questions after an incident. The operational test is whether AI actions remain attributable after delegation, not whether the gateway produced a log line.

The most useful programmes will stop asking whether AI is allowed and start asking which identities, tools, and datasets are governed under explicit policy. That is where AI gateway design meets NHI lifecycle management, and where most current control gaps become visible.

For practitioners

• Inventory every AI traffic path Classify developer-built apps, desktop tools, consumer web apps, and embedded SaaS AI separately, then assign the control that can actually enforce policy for each path. Where interception is impossible, rely on endpoint, device, or vendor-admin controls instead.
• Bind AI activity to accountable identities Use scoped credentials, per-application tokens, and token exchange so the acting identity is preserved across chained calls. Do not allow shared API keys to hide which user, app, or agent triggered the request.
• Separate discovery from enforcement Log prompts, responses, identity, and cost into one audit stream, then export that stream to your SIEM for reconstruction and compliance evidence. Visibility without correlated evidence is not governance.
• Treat MCP tools as privileged action surfaces Apply tool-level allowlisting, not whole-server access, and require outbound authentication controls for each server connection. A tool that can write or send must be governed like any other high-impact NHI.

Key takeaways

• AI gateway value depends on governance coverage, not just low latency or routing convenience.
• Shadow AI and agent-driven access create identity and audit problems that traditional perimeter controls do not resolve.
• Practical deployment requires per-path enforcement, preserved identity, and correlated logs that stand up in incident response.

Key terms

• AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Token Exchange: Token exchange is an OAuth pattern that swaps one credential or token for another with narrower scope or different trust context. In NHI governance, it is useful when a workload must cross boundaries without carrying broad, reusable privileges into downstream systems.
• MCP Gateway: The control layer that relays assistant intent to tools and data sources through the Model Context Protocol. In practice, it becomes a policy boundary, not just a transport layer. If it trusts model output too early, it can turn unverified reasoning into real-world execution or disclosure.

What's in the full article

TruFoundry's full blog post covers the operational detail this post intentionally leaves for the source:

• Specific gateway deployment patterns for developer apps, desktop clients, consumer web apps, and SaaS-embedded AI features.
• Implementation detail on Virtual Account Tokens, device certificates, and token exchange across chained agent calls.
• Step-by-step guidance for integrating the gateway with MDM, SWG, SIEM, and endpoint controls.
• Practical examples of how MCP Gateway policy separates inbound authentication from outbound tool access.

👉 TruFoundry's full post covers the gateway control paths, identity handling, and audit mechanics in more detail.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.