Notifications
Clear all
Topic starter
20/08/2026 6:01 pm
TL;DR: Built-in Topics facets such as Task, Sentiment, and Issues can be too coarse for production AI agent review, and custom facets can turn recurring questions into stable, workflow-linked labels for dashboards and evals, according to Braintrust. The practical issue is governance: without narrowly scoped, repeatable classifications, teams cannot reliably track quality, failure modes, or product signals across traces.
NHIMG editorial — based on content published by Braintrust: How to design custom facets for AI agent traces (2026)
Questions worth separating out
Q: How should teams design custom facets for AI agent traces?
A: Start with one operational question, define mutually exclusive labels, and make sure each label routes to a concrete workflow.
Q: Why do broad trace labels fail in AI agent governance?
A: Broad labels collapse too many behaviours into the same bucket, so the classifier cannot separate meaningful patterns from noise.
Q: What do security teams get wrong about observability labels for AI agents?
A: They often treat labels as reporting categories instead of decision tools.
Practitioner guidance
- Define one review question per facet Start with a single operational question such as churn risk, tool reliability, or citation quality, then build labels only for that decision.
- Use the narrowest trace slice that contains the signal Choose user-only, error-window, or metadata-projected input based on the label you need.
- Tie every label to a downstream workflow Make sure each label routes to a concrete next step, such as review, escalation, filtering, or scoring.
What's in the full article
Braintrust's full article covers the implementation detail this post intentionally leaves at the governance level:
- Preprocessor patterns for user-only traces, failure windows, and metadata projection in production facets
- Prompt structures and label boundaries that improve consistency across repeated reruns
- Exclusion regex handling for NONE and N/A outputs so empty classifications do not pollute clustering
- Worked examples for support, coding, multilingual chatbots, and sales or PLG agents
👉 Read Braintrust's guide to designing custom facets for AI agent traces →
AI agent trace facets: when built-in labels are too broad?
Explore further
20/08/2026 6:39 pm
Custom facets are really governance controls for AI agent observability. The article shows that classification quality depends on whether the team can define a narrow, repeatable question before it tries to automate review. That is a governance pattern, not just a product feature, because the label taxonomy becomes the control surface for dashboards, evals, and alerts. In agentic AI environments, poor trace classification creates blind spots similar to weak identity telemetry. Practitioners should treat facet design as part of operational governance, not a reporting convenience.
A question worth separating out:
Q: How do you know if a custom facet is actually working?
A: A facet is working when the same trace gets the same label on rerun, the labels are easy to interpret, and each output clearly maps to a downstream workflow. If clusters are noisy or labels overlap, the taxonomy or preprocessor needs to change before the facet can be trusted.
👉 Read our full editorial: Custom facets make AI agent trace review operationally useful
