TrueFoundry and TrojAI define a secure AI gateway stack

At a glance

What this is: This is a partnership analysis of an AI gateway plus runtime defense stack that combines central traffic control with inline policy enforcement for GenAI workloads.

Why it matters: It matters because IAM and security teams need governance that reaches beyond model access into request-level control, especially as AI agents and applications move into production.

Context

AI gateway governance is the control plane for how requests reach models, while runtime defense decides what those requests are allowed to do once they arrive. The gap appears when organisations centralise routing and logging but leave prompt injection, data leakage, and unsafe output handling to downstream tools that cannot see the full request lifecycle.

For AI agents and GenAI applications, that gap is an identity and policy problem as much as a security problem. The article’s core point is that access control, observability, and runtime enforcement need to be designed together if teams want consistent production governance across multiple model providers.

The deployment pattern described here is typical of current AI programmes: visibility arrives before enforceable policy maturity, and governance fragments across gateway, proxy, model, and application layers.

Key questions

Q: How should security teams govern AI gateways in production environments?

A: Security teams should govern AI gateways like shared control planes, not convenience proxies. That means tying every key, token, and routing policy to an owner, enforcing least privilege for configuration changes, and making logging, retention, and fallback behaviour auditable. The gateway should sit inside IAM, secrets, and incident response processes, not beside them.

Q: Why do AI gateways need runtime enforcement as well as observability?

A: Observability tells you what the AI system did, but it does not prevent prompt injection, leakage, or unsafe output in the moment. Runtime enforcement adds the ability to block, redact, or pass traffic before harmful content reaches the model or exits the system. Without that inline layer, policy remains retrospective rather than protective.

Q: What are the signs that AI governance is too fragmented to support scale?

A: Common signs include duplicated governance tools, inconsistent policies across business units, and teams managing AI or data controls in isolation. Another warning is when organisations cannot confidently explain how data is governed across platforms. If governance only exists in pockets, scaling AI usually creates more risk, not less, because controls fail to follow the use case end to end.

Q: When should teams treat AI agents as a separate governance surface?

A: Teams should do so as soon as AI agents can route requests, call models, or handle sensitive data without direct human approval for each action. At that point, the agent’s access, policy boundaries, and output handling need explicit governance just like any other privileged non-human identity.

How it works in practice

How AI gateways centralise access without enforcing content policy

An AI gateway sits in front of model providers and governs request flow. It can authenticate callers, apply rate limits and quotas, route traffic by latency or cost, and log tokens, failures, and usage patterns. That creates a useful control point for inventory and oversight, but it does not by itself inspect the semantic content of prompts or responses deeply enough to stop malicious instructions, secrets exposure, or unsafe completions. In practice, the gateway solves traffic governance while leaving content governance unresolved unless a second layer participates in the request path.

Practical implication: treat gateway control as necessary infrastructure, not as a substitute for runtime policy enforcement.

Why runtime enforcement changes the AI security boundary

Runtime enforcement proxies inspect prompts and responses as traffic moves through the stack. That lets policy decide whether to block, redact, or pass a request based on malicious intent, sensitive data, toxicity, or jailbreak patterns. This is materially different from post hoc logging because the decision happens before the model response becomes an exposure event. For production AI, that shifts security from visibility after the fact to control in the moment, which is the only point where prompt injection and data leakage can still be contained.

Practical implication: place policy checks in the live request path if the goal is to stop harmful output before it is returned.

How gateway routing and defence proxies divide responsibilities

The clean architectural pattern is separation of responsibilities. One layer governs who can send traffic, which provider receives it, and how the request is routed. The other layer governs whether the content and response are acceptable under policy. That separation is useful because AI security failures are not only about access, they are also about what an authorised request can induce the model to reveal or generate. Without both layers, organisations either over-restrict access or under-control content, and neither outcome gives stable production governance.

Practical implication: define which control owns access, which owns content inspection, and where policy decisions terminate.

NHI Mgmt Group analysis

AI gateway visibility is not governance: Centralised routing, logging, and quota enforcement solve only the access side of the problem. They do not answer whether a request should be blocked, redacted, or allowed once it has reached the model. The governance failure in most AI stacks is assuming that observability equals control; practitioners should treat those as separate functions.

Runtime policy is the control layer that decides whether AI output is safe enough to return: Prompt injection, secret leakage, and unsafe content are request-time problems, not dashboard problems. Once the model has produced an unsafe response, logging can explain the event but cannot contain it. AI programme owners should therefore evaluate enforcement in the live path, not after the fact.

Unified control and enforcement is the named concept emerging here: the stack works when request governance and response governance are designed as one operating model. That is the practical breakpoint between experimental AI usage and production AI governance, and it is especially important when AI agents are involved because agent-driven traffic can scale policy mistakes very quickly.

Agentic AI changes the meaning of access scope: permissions granted to an AI application can be exercised at machine speed across multiple requests, models, and providers. That makes gateway policy, routing policy, and response filtering part of the same governance surface. The implication is that identity and access teams must stop treating model access as a simple allow-or-deny decision.

OWASP-NHI and OWASP-AGENTIC both matter here: the stack is managing non-human access into model services while also defending against agent-driven misuse of that access. That combination requires security teams to think about credential scope, request intent, and content safety together. Practitioners should map the control split before production rollout, not after the first policy exception.

From our research library:

• 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
• Systems with least-privileged AI access had a 17% incident rate vs 76% for over-privileged systems. Organisations failing to scope AI access properly are 4.5x more likely to experience a security incident, according to the 2026 Infrastructure Identity Survey.
• Read next: AI Agent Authorisation Guide

What this signals

Unified control and enforcement is the concept practitioners should watch closely as AI stacks move from pilot to production. Gateway visibility, provider routing, and telemetry reduce chaos, but they do not close the policy gap unless inspection and blocking happen before a model response is released.

The programme implication is straightforward: AI governance must be designed around the request lifecycle, not around a single integration point. Once AI agents are able to move traffic across multiple providers, every exception becomes a security decision, not an operational convenience.

For practitioners

• Define separate owners for access and content control Assign the gateway team responsibility for authentication, routing, quotas, and telemetry, while a second control owns prompt and response inspection, blocking, and redaction.
• Inspect requests before model execution Place prompt injection, jailbreak, and sensitive data detection in the live request path so policy can stop harmful traffic before a model response is generated.
• Map AI traffic by model, provider, and policy outcome Track which agents and applications can reach which providers, what policy decisions are applied, and where exceptions are created in production.
• Review AI agent access as a governance surface Treat each AI agent or application as a non-human identity with scoped privileges, observable routes, and explicit enforcement boundaries.
• Test response filtering against unsafe completion paths Validate that toxic output, data leakage, and adversarial responses are blocked or redacted after generation but before they reach users or downstream systems.

Key takeaways

• AI gateway visibility improves oversight, but it does not by itself stop prompt injection, data leakage, or unsafe model outputs.
• The article’s core security point is that access control and runtime enforcement solve different halves of the AI governance problem.
• For production AI, teams need an identity-aware control plane and an inline policy layer if they want repeatable governance across models and agents.

Key terms

• AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
• Runtime Enforcement: Runtime enforcement is the practice of blocking malicious behaviour while software is running, rather than only detecting it after the fact. It monitors process activity, network actions, and privilege changes so a live attack can be interrupted at the point of execution.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Non-Human Identity (NHI): A digital identity assigned to a non-human entity such as a software application, service account, API key, bot, machine, or AI agent that enables it to authenticate and interact with systems without direct human involvement. NHIs now outnumber human identities in most enterprises by 25 to 50 times.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.