Braintrust reviews expose a bigger gap in AI governance

At a glance

What this is: Braintrust reviews point to a strong AI evaluation platform, but TruFoundry says it does not govern request-path access, tool control, or spending limits.

Why it matters: AI teams need to separate post-inference observability from governance controls that decide who and what can reach the model before execution.

Context

Evaluation platforms help AI teams measure quality changes after prompts, models, or retrieval logic are altered, but they do not by themselves govern who can invoke a model or what a request is allowed to do. In practice, that leaves a security and governance gap between analysis of model output and control of model access. For enterprise AI programmes, the distinction matters because request-path policy is where identity, budget, and tool-use decisions belong.

The article frames Braintrust as an evaluation and observability layer rather than an AI gateway. That boundary is especially relevant where agents, private prompts, and MCP-connected tools are involved, because the controls that constrain those actions sit before inference, not after it. TruFoundry’s analysis is therefore about governance scope, not product quality in isolation.

Key questions

Q: What breaks when AI evaluation tools sit downstream of inference instead of on the request path?

A: They can show what a model produced, but they cannot prevent the call, restrict the tool, or stop the spend before execution. That means teams may improve visibility while leaving access, authorisation, and cost controls unmanaged. The control must exist where the request is still stoppable, not where the trace is already complete.

Q: Why do AI gateways matter when teams already have traces and evals?

A: Because traces and evals answer whether output quality changed, while gateways answer whether the request should have been allowed at all. A gateway can enforce identity, routing, budgets, and tool policy before the model call. Without that layer, observability documents behaviour but does not govern it.

Q: What are the signs that an AI evaluation platform is being used as a governance control?

A: Warning signs include relying on post-run alerts to manage spend, assuming trace logs can approve tool execution, or using evaluation scores as a proxy for access decisions. Those patterns show the organisation has visibility without enforcement. A mature design keeps measurement and policy control in separate layers.

Q: Should enterprises place MCP tool approvals in the model layer or the gateway layer?

A: The gateway layer is the better place because it sits before execution and can evaluate caller identity, requested tool, and policy context in one decision. Putting approvals in the model layer is too late if the tool call has already been formed. Tool governance should be treated as privileged access control, not post hoc review.

Technical breakdown

Why post-inference observability cannot enforce request policy

Evaluation and observability platforms inspect traces, scores, latency, and prompt variants after requests have already reached a model. That makes them useful for quality assurance, debugging, and regression analysis, but not for deciding whether the request should be allowed in the first place. Request-path governance needs a control point that can authenticate the caller, evaluate policy, and either permit, constrain, or reject the call before inference executes. In AI estates, that control point is where identity, budget, and tool permissions become enforceable rather than merely visible.

Practical implication: Treat observability as evidence after execution, not as the enforcement layer for model access or tool use.

AI gateway controls versus evaluation workflows

An AI gateway sits in front of model calls and can apply role-based access, request filtering, budget ceilings, routing logic, and tool policy before traffic reaches the provider. An evaluation platform can replay traces, compare outputs, and help teams understand whether a model change improved quality. These are complementary functions, but they solve different problems. When teams collapse them into one category, they risk assuming that better telemetry also means stronger governance. It does not. The gateway owns the checkpoint; the evaluation layer owns the analysis.

Practical implication: Separate the control plane from the measurement plane when designing enterprise AI operating models.

MCP tool control is a governance boundary, not a logging problem

Model Context Protocol links AI systems to external tools and data sources, which means a model call can quickly become an action against another system. Once tool use is in play, the security question is no longer only what the model produced, but whether the caller was authorised to trigger that tool at all. Logging the event later can support forensics, but it cannot prevent an unsafe action or an over-privileged agent from executing it. That makes tool governance a pre-execution decision problem tied to identity and privilege.

Practical implication: Place MCP approvals and tool restrictions on the request path, not only in downstream audit logs.

Threat narrative

Attacker objective: The objective is to use permitted AI access to trigger unintended model, tool, or spending behaviour before any policy enforcement stops it.

1. Entry occurs when a user, service, or agent reaches an AI model without a front-door policy check on identity, budget, or tool scope.
2. Escalation follows when the same request path is reused to call external tools or repeat expensive inference without hard limits.
3. Impact appears as uncontrolled spend, unsafe tool execution, or data exposure that observability records only after the action has happened.

Breaches seen in the wild

• LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI evaluation is not AI governance: braintrust-style observability tells teams what happened inside inference, but it does not decide whether a request should have been allowed. That leaves a structural gap in enterprise AI programmes: the system that measures quality is not the system that constrains action. The governance lesson is that control must exist before execution, not only after telemetry lands.

Request-path policy is the missing control plane: enterprises need a place to enforce identity, budget, and tool-use rules before a model call executes. Without that checkpoint, teams may have excellent traces and still no effective limit on who can invoke models, what tools an agent can use, or how much spend a workflow can generate. The practical conclusion is to design AI governance around enforcement points, not dashboards.

MCP governance gap: when AI systems can invoke external tools, the security boundary shifts from prompt quality to authorisation of action. That makes tool control an identity problem as much as an AI problem, because the entity making the call may be a human, a service, or an agent. Practitioners should treat MCP access as privileged capability that requires explicit request-path governance.

Enterprise review noise can hide the real architecture issue: positive reviews about evals and tracing can obscure whether a platform can protect regulated workloads. The key question is not whether the output pipeline is measurable, but whether the request pipeline is governable under identity, compliance, and cost constraints. Teams should benchmark products against where enforcement actually happens, not where visibility is easiest.

Named concept, request-path governance gap: the gap is the difference between seeing AI behaviour and constraining AI behaviour. Braintrust-style platforms reduce uncertainty after inference, while enterprise controls must stop unauthorised calls, tool actions, or runaway budgets before they start. Practitioners should map every AI control to the stage where it can still change the outcome.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Authorisation Guide

What this signals

Request-path governance gap: AI programmes increasingly fail at the point where identity, budget, and tool permission decisions must be enforced before execution. A platform that measures output quality after inference is useful, but it leaves the operational boundary unchanged, so governance must move to the request path if the organisation wants actual control.

The practical implication for enterprise teams is that evaluation tooling and enforcement tooling need different owners, different control objectives, and different audit expectations. Where MCP-connected tools or agentic workflows are involved, that separation becomes material because the same request can trigger both model inference and privileged action.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.

For practitioners

• Define the request-path control point Place model access decisions, tool approvals, and spend limits in a gateway or equivalent checkpoint before inference executes, not in the tracing layer.
• Separate evaluation from enforcement Use traces, evals, and regression tests for quality assurance, but do not treat them as substitutes for identity-aware policy enforcement.
• Classify MCP-connected tools as privileged access Review which tools agents can reach, then require explicit authorisation for each tool class and each execution context.
• Set hard budget controls ahead of execution Reject or throttle requests before provider calls when token spend, batch jobs, or agent loops exceed approved thresholds.
• Test enterprise fit against deployment and identity needs Check whether self-hosting, SSO, RBAC, retention, and audit requirements are available at the tier you would actually deploy.

Key takeaways

• Braintrust reviews point to strong evaluation and tracing value, but the real governance gap is that those controls sit after inference and cannot stop unauthorised calls.
• Enterprise AI programmes need a request-path checkpoint for identity, budgets, and tool use if they want to enforce policy before a model acts.
• MCP-connected tools turn AI governance into a privileged access problem, which means enforcement has to happen before execution rather than in retrospective logs.

Key terms

• Request-path Governance: Request-path governance is the set of policies that apply before an AI request executes. It covers who can call a model, which tools can be reached, what data can move, and whether the request should be blocked, throttled, or routed differently.
• AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
• MCP Tool: An MCP tool is a capability exposed through the Model Context Protocol so an AI agent can call external functions, retrieve data, or trigger actions. Technically, it is a structured interface with defined inputs, outputs, and permissions, allowing controlled interaction between an agent runtime and systems such as databases, APIs, or workflows.
• Post-inference observability: Post-inference observability is the practice of tracing and analysing model behaviour after a request has already executed. It is valuable for debugging, quality review, and regression testing, but it cannot itself enforce access, budgets, or tool policy before the model responds.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and secrets management for teams designing control points around AI systems. It helps security and platform practitioners connect identity decisions to the broader programme they are responsible for.