Topic starter
19/08/2026 3:53 pm
TL;DR: OpenTelemetry gives LLM and agent teams a vendor-neutral way to trace prompts, retrieval, tool calls, token usage, and agent execution across multiple backends, while Braintrust layers evaluation on the same spans so teams can score output quality and production behaviour, according to Braintrust. The core issue is that standard APM can show healthy latency and errors while missing whether an answer was grounded, safe, or correct.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “OpenTelemetry for LLM tracing: a guide to instrumenting agents and routing spans anywhere”.
Key questions
Q: What breaks when teams monitor agents only with standard APM?
A: Standard APM can show that an agent request completed quickly and without transport errors, but it cannot prove that the answer was grounded, policy-compliant, or safe to use.
Q: Why does clean core matter for identity and access governance?
A: Clean core matters because it changes where controls can live.
Q: How do security teams know if agent observability is actually working?
A: Observability is working only when teams can tie together token activity, tool calls, and latency for a specific agent session.
Practitioner guidance
- Instrument prompts, retrieval, tool calls, and outputs as one trace Capture the full execution path so model calls, context retrieval, tool usage, and final responses can be reviewed together.
- Separate operational monitoring from output evaluation Keep latency, error rates, and service health in your APM stack, but score the same spans for groundedness, policy alignment, and task success in a quality layer.
- Use the Collector to route one trace to multiple backends Forward the same GenAI spans to your observability platform and your evaluation backend through a collector pipeline so you can inspect system health and model quality from one source of truth.
Bottom line: Standard monitoring can confirm that an LLM service is healthy without showing whether the answer was grounded, safe, or correct.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
19/08/2026 4:30 pm
Agent observability is becoming a governance control, not just an engineering aid. Once model calls, retrieval, and tool execution sit inside one request path, the trace becomes part of the control evidence for delegated action. That means the question is not only whether the service responded, but whether the agent used authorised data and tools in a way the business can trust. For practitioners, observability now has to support governance review as well as debugging.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should organisations keep tracing and evaluation in the same workflow?
A: Yes, because tracing without evaluation only explains what happened, while evaluation without trace context cannot explain why it happened. Keeping them connected lets teams inspect the run, judge the result, and preserve the failure as evidence for the next release.
👉 Read our full editorial: OpenTelemetry for LLM tracing exposes the gap in agent observability