Topic starter
21/08/2026 4:49 pm
TL;DR: Local red teaming and production-linked evaluation are splitting apart, with Braintrust in the middle of that divide through tracing, release gating, and human review. AI governance has to span tool calls, retrieval steps, and regression checks or production risk will outrun developer-only testing.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “Braintrust vs. Promptfoo: 2026 LLM evaluation comparison”.
Key questions
Q: Why do pre-deployment tests fail to manage production AI risk?
A: Because they only measure behaviour before release.
Q: When should organisations block AI releases with evaluation gates?
A: Organisations should block AI releases when the system’s behaviour is safety-critical, customer-facing, or tied to governed enterprise workflows.
Q: How should teams evaluate AI systems when the model uses tools or multi-step workflows?
A: Teams should evaluate the full trace, not only the final output, when AI systems call tools or run multi-step workflows.
Practitioner guidance
- Define the production evaluation boundary Map which AI workflows are allowed to ship with local-only tests and which require live tracing, scorer reuse, and release gating before merge.
- Tie CI/CD checks to release policy Use automated thresholds that block merges when quality, safety, or security scores fall below the minimum required for deployment.
- Promote production failures into regression tests Turn real trace failures into reusable evaluation cases so the same defect cannot reappear in the next build.
Bottom line: AI evaluation splits into two models here, one centred on local red teaming and one centred on production traces, and the difference changes how release control is enforced.
Glossary terms: Red Teaming, Production Tracing, Approval Gate, Human Review
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 3 days ago by NHI Mgmt Group
21/08/2026 5:07 pm
AI evaluation debt is now a governance problem, not just a testing problem. Once teams ship AI into production, evaluation has to follow the release path, not sit beside it. A local YAML test suite can still be useful, but it cannot govern live traces, live scores, and release enforcement at scale. The practitioner conclusion is straightforward: evaluation that does not touch production is incomplete governance.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
- 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools, according to the Ultimate Guide to NHIs.
A question worth separating out:
Q: What should security teams do after an AI production failure is discovered?
A: Security teams should turn the failure into a reusable regression test, preserve the trace that shows what happened, and make the same case part of the next deployment gate. That converts an isolated incident into a control improvement and keeps the issue from reappearing unnoticed in later releases.
👉 Read our full editorial: AI evaluation gaps widen when CI/CD, tracing, and red teaming split