Topic starter
20/08/2026 6:01 pm
TL;DR: Built-in Topics facets such as Task, Sentiment, and Issues can be too coarse for production AI agent review, and custom facets can turn recurring questions into stable, workflow-linked labels for dashboards and evals, according to Braintrust. The practical issue is governance: without narrowly scoped, repeatable classifications, teams cannot reliably track quality, failure modes, or product signals across traces.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “How to design custom facets for AI agent traces (2026)”.
Key questions
Q: What should teams do when built-in AI agent trace labels are too broad?
A: Create a custom facet only when the team keeps asking the same operational question and the built-in labels cannot answer it at the right level of detail.
Q: Why do broad trace labels fail in AI agent governance?
A: Broad labels collapse too many behaviours into the same bucket, so the classifier cannot separate meaningful patterns from noise.
Q: What are the signs that a custom facet taxonomy is failing?
A: The most common signs are heavy use of catch-all labels, overlapping categories, inconsistent results on rerun, and clusters that do not map cleanly to any workflow.
Practitioner guidance
- Design facets around one decision point Anchor each custom facet to a single operational question such as refund disposition, tool reliability, or churn risk, and avoid label sets that mix unrelated dimensions like tone and outcome.
- Narrow the preprocessor to the evidence slice Pass only the trace spans needed for the facet, such as user messages, error windows, or selected metadata, so the classifier sees consistent evidence on every run.
- Use mutually exclusive labels with explicit workflow routing Give each label one downstream action, such as review, escalation, filtering, or scoring, so the taxonomy supports a concrete operating process instead of general tagging.
Bottom line: Built-in trace facets are useful for broad triage, but they are often too coarse to support the recurring review questions production AI agents generate.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
20/08/2026 6:39 pm
Custom facets are a governance control, not just an analytics convenience. When AI agent traces become operational evidence for quality, routing, or escalation, the classification layer is part of the control environment. Broad labels create false confidence because they look measurable while hiding the distinctions practitioners actually need. The discipline here is to design the taxonomy around decision points, not around whatever the model can label loosely.
A question worth separating out:
Q: How should AI teams decide whether a custom facet belongs in production?
A: Treat production readiness as a workflow test, not a taxonomy test. A facet belongs in production when it classifies a narrow slice consistently, feeds a concrete action such as routing or scoring, and still behaves the same after a rerun on historical traces. If any of those conditions fail, it is still experimental.
👉 Read our full editorial: Custom facets make AI agent trace review operationally useful