AI gateway guardrails are shifting pricing and governance decisions

At a glance

What this is: TruFoundry argues that AI gateway guardrails are becoming a pricing and governance decision because buyers must choose between a built-in control layer and a separate security product.

Why it matters: This matters to IAM and AI security teams because guardrail placement changes who governs access, what gets logged, and how much operational complexity a production AI stack carries.

By the numbers:

• Lasso Security describes its red-teaming library as covering 3,000+ attack types and techniques.
• TrueFoundry says its gateway can handle 350+ RPS on just 1 vCPU.
• TruFoundry states that TrueFoundry runs at ~3-4 ms latency while Lasso reports sub-50ms decision latency.
• TruFoundry says Lasso reports a 98.6% detection accuracy rate.

Context

AI gateway guardrails are the policy and enforcement checks that sit between users, models, tools, and data. In practice, the governance question is not whether guardrails exist, but where they live, who owns them, and how much of the AI stack they must inspect to be effective.

This article is really about control-plane design for production AI. When runtime enforcement, discovery, and audit reporting are split across separate products, teams inherit more integration work and more decision points over authentication, tool access, and policy consistency across agentic workloads.

Key questions

Q: How should security teams decide whether AI guardrails belong in the gateway or in a separate layer?

A: Teams should place guardrails where they can actually enforce policy at runtime with the least duplication. If the gateway already controls routing, authentication, and observability, native guardrails often reduce integration work and improve auditability. Separate layers make sense only when they add a materially different control capability or when existing gateways cannot inspect the relevant requests.

Q: Why do separate AI guardrail products often create more governance complexity?

A: Because they introduce another policy engine, another log source, and another place where identity and tool access can diverge. In agentic environments, that makes it harder to prove which actions were authorised, which were blocked, and which system owned the decision. Complexity rises fastest when enforcement, inventory, and audit evidence are not aligned.

Q: What breaks when BOM data is not connected to runtime enforcement?

A: The BOM becomes stale almost immediately. Teams may believe a component is approved because it was documented at build time, while the running workload has drifted, changed dependencies, or inherited new policy violations. That gap creates audit risk, compliance blind spots, and false confidence in release assurance.

Q: What should procurement teams compare besides the sticker price of an AI gateway?

A: They should compare the full governance cost, including integration effort, policy maintenance, logging retention, audit evidence production, and the number of tools needed to cover the same control scope. A lower licence fee can still produce a higher operating cost if guardrails and observability sit in separate products.

Technical breakdown

Gateway-native guardrails versus add-on enforcement

AI gateways commonly handle routing, authentication, and observability, while a separate guardrail layer inspects prompts, outputs, and tool calls for policy violations. When guardrails sit outside the gateway, the security team gets a second enforcement point, a second logging surface, and a second vendor workflow to govern. When they are native to the gateway, policy enforcement can be applied earlier in the request path and correlated with routing and identity decisions. The technical tradeoff is not simply feature overlap. It is whether the control plane is one policy engine with shared context or two products that must be kept logically aligned.

Practical implication: Map every AI control to the layer that can actually enforce it before buying a separate guardrail product.

Why discovery and AI-BOM data change the governance burden

Discovery and AI-BOM functions inventory agents, tools, and MCP servers so teams can see what their AI estate is calling at runtime. That matters because policy is only as good as the inventory behind it. If the gateway does not know which tools, agents, or external services are in use, guardrails become reactive rather than preventive. In agentic environments, this is especially important because tool chains can expand dynamically, and governance has to keep pace with what the system is allowed to call. The operational question is whether inventory feeds the same policy layer that enforces access, or whether teams are reconciling two partially overlapping views of the estate.

Practical implication: Tie AI-BOM discovery to enforcement decisions so unmanaged tools do not become invisible policy exceptions.

Auditability depends on where the decision is made

Audit-ready AI governance requires more than alerts. Teams need to know which policy fired, which identity or workload made the request, what tool or MCP server was involved, and whether the action was blocked or allowed. If runtime enforcement is separate from the gateway, investigators have to reconstruct the path across products after the fact. That weakens evidence quality for compliance and incident review. A gateway that natively couples authentication, policy, routing, and observability can produce a clearer chain of control, but only if policy decisions are logged in a way that supports later review. In AI operations, the location of the decision often matters as much as the decision itself.

Practical implication: Require a single auditable decision trail for model, tool, and agent requests before standardising on a governance stack.

Threat narrative

Attacker objective: The attacker objective is to reach tools, data, or downstream actions through an AI workflow that is not governed consistently at runtime.

1. Entry occurs when an AI application, agent, or MCP-connected workflow sends a request through the gateway and the control layer must decide what can proceed.
2. Credential and policy abuse follow when tool access or model routing is insufficiently constrained, allowing an agent to invoke services outside intended scope.
3. Impact is governance drift, where security teams lose a clear view of which AI actions were authorised, blocked, or delegated across the stack.

Breaches seen in the wild

• LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
• Hugging Face API tokens exposed 2023: Lasso Security found 1,681 live Hugging Face tokens in public code, 655 with write access, reaching 723 organisations including Meta; all were revoked.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI gateway guardrails are becoming a control-plane decision, not a point product decision. The real question is whether policy, routing, identity, and observability are governed in one place or split across multiple products. That split increases the chance of policy drift, duplicated logging, and inconsistent enforcement across models, tools, and agents. Practitioners should treat guardrail placement as architecture, not procurement.

Discovery is now part of enforcement for agentic AI. Once gateways must understand agents, tools, and MCP servers, inventory becomes a prerequisite for policy rather than a separate management task. That connects directly to NHI governance because tool credentials, service accounts, and delegated access are what make AI actions possible in the first place. If the inventory is incomplete, the control is incomplete.

Runtime AI governance increasingly resembles NHI control, even when the article is framed as pricing. The policy question is who or what is authorised to act, what it can call, and how long that authority persists. That means AI gateway architecture now intersects with machine identity, secrets handling, and privileged tool access. Practitioners should align gateway controls with identity governance, not treat them as a separate island.AI governance debt: this is the accumulation of incomplete inventory, duplicated policy engines, and disconnected audit trails across AI products. The more teams add point controls without a unified enforcement model, the harder it becomes to prove what happened in production. Practitioners should reduce the number of places where AI policy can diverge.

Pricing transparency changes buying behaviour, but governance fit still decides the outcome. The article shows that enterprise buyers are not only comparing capabilities, they are comparing operating models. That pushes teams to ask whether a separate guardrail product just adds another contract and another policy boundary. Practitioners should benchmark control placement against their audit and lifecycle requirements, not just their budget.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

AI governance debt: the more teams split routing, guardrails, discovery, and audit into separate products, the more likely they are to inherit policy drift and evidence gaps. A gateway that natively owns policy still needs rigorous logging, but it reduces the number of control seams that can fail.

For identity teams, the AI gateway is becoming a machine identity boundary as much as a traffic boundary. Tool credentials, delegated access, and MCP governance now sit in the same decision path, so the control model has to account for both runtime requests and the identities that make those requests possible. Organizations cannot treat that as a purely application-layer issue.

For practitioners

• Define the enforcement boundary before procurement Decide whether prompt, tool, and output controls belong in the gateway, in a separate security layer, or in both, and document which layer owns each decision type.
• Inventory agents, tools, and MCP servers continuously Maintain a living AI-BOM so every agentic application, external tool, and MCP dependency is mapped to an owner, policy, and approval path.
• Unify identity and policy logging Ensure authentication events, policy decisions, and tool-call records can be correlated across the gateway and any add-on guardrail platform.
• Price total governance overhead, not just product licence cost Include integration effort, logging retention, policy maintenance, and audit evidence production when comparing a native gateway model with a separate security layer.

Key takeaways

• AI gateway guardrails are no longer just a feature comparison, because placement affects who enforces policy, who can audit it, and how much operational overhead the AI stack carries.
• Separate guardrail layers can increase control visibility, but they also add policy drift risk when discovery, authentication, and enforcement are not tied to the same control plane.
• For practitioners, the key decision is whether native gateway guardrails cover the required governance scope or whether a second layer genuinely adds a control that cannot be replicated in the gateway.

Key terms

• AI Gateway Guardrails: Policy checks applied at an AI gateway to filter prompts, constrain tool use, and inspect outputs before requests continue. In enterprise settings, they function as a control layer for routing, authentication, observability, and content safety across models and agents.
• AI-BOM: An AI bill of materials is a structured inventory of the components that define an AI agent, including the model, prompt, tools, retrieval sources, and dependencies. In practice, it is the evidence base for review, change control, and risk assessment when the agent evolves after deployment.
• Runtime Enforcement: Runtime enforcement is the practice of blocking malicious behaviour while software is running, rather than only detecting it after the fact. It monitors process activity, network actions, and privilege changes so a live attack can be interrupted at the point of execution.
• MCP governance: MCP governance is the set of policies, controls, and oversight used to manage how AI agents use the Model Context Protocol to reach tools and data. It defines approved connections, authentication, authorization, logging, data handling, and change control so agent actions remain traceable, bounded, and aligned with security and compliance requirements.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management. It helps practitioners connect AI governance decisions to the identity and access controls that make those systems work.