Notifications
Clear all
Topic starter
20/08/2026 6:05 pm
TL;DR: LLM production failures often look successful in observability layers, so hallucinations, retrieval misses, tool-call errors, and schema violations can recur unless teams promote real traces into regression datasets, according to Braintrust. The governance problem is not detection alone but whether production evidence becomes a reusable control that blocks repeat failures before release.
NHIMG editorial — based on content published by Braintrust: How to turn LLM production failures into regression tests
By the numbers:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes, and as quickly as 9 minutes in some cases.
Questions worth separating out
Q: What breaks when production failures are not turned into regression tests?
A: The same defect keeps reappearing because the organisation has evidence of failure but no durable control to prevent recurrence.
Q: Why do production AI traces create security and governance risk?
A: Production traces can include prompts, tool calls, model outputs, and evaluation signals, which means they may expose sensitive context beyond standard application logs.
Q: How do security teams know a regression scorer is working?
A: A scorer is working when it consistently fails the known bad case and passes the known good case under the same conditions.
Practitioner guidance
- Preserve full production traces for every diagnosed failure Store the input, output, intermediate tool calls, retrieved context, prompt version, model version, and runtime metadata so the failure can be reproduced later.
- Label the failure mode before you promote the span Use a consistent taxonomy such as hallucination, retrieval miss, tool argument error, instruction-follow failure, or format violation so similar defects can be clustered.
- Convert recurring failures into versioned regression rows Promote representative traces into a dataset only after review, and keep source span IDs in metadata for traceability across releases and incidents.
What's in the full article
Braintrust's full article covers the operational detail this post intentionally leaves for the source:
- Step-by-step dataset promotion workflows from Logs and BTQL for failed production spans.
- Example scorer patterns for hallucination, tool-call errors, schema violations, and instruction-follow failures.
- Human review and thresholding guidance for deciding which traces should block a release.
- Implementation notes for preserving prompt version, model version, and source-span traceability.
👉 Read Braintrust's guide to turning LLM production failures into regression tests →
LLM production failures to regression tests: what teams keep missing?
Explore further
20/08/2026 6:40 pm
Production traces are becoming the new control plane for AI quality. LLM failures that only exist in observability dashboards are too easy to forget, reintroduce, or misclassify. The real control is whether a failed span becomes a persistent regression case with a scorer attached. That is a governance shift, not just a testing tactic, because it links production evidence to release gating and auditability.
A question worth separating out:
Q: Who should own AI regression tests when agents touch tools and data?
A: Ownership should sit with the team responsible for the model workflow, but governance should include AI security, application engineering, and the control owner for any sensitive data or privileged tool access. When an agent can call tools or retrieve restricted information, the regression suite is part of the control environment, not just the test harness.
👉 Read our full editorial: LLM production failures should become regression tests, not ad hoc fixes
