Notifications
Clear all
Topic starter
19/08/2026 3:53 pm
TL;DR: OpenTelemetry gives LLM and agent teams a vendor-neutral way to trace prompts, retrieval, tool calls, token usage, and agent execution across multiple backends, while Braintrust layers evaluation on the same spans so teams can score output quality and production behaviour, according to Braintrust. The core issue is that standard APM can show healthy latency and errors while missing whether an answer was grounded, safe, or correct.
NHIMG editorial — based on content published by Braintrust: OpenTelemetry for LLM tracing: a guide to instrumenting agents and routing spans anywhere
By the numbers:
- 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools.
- Only 5.7% of organisations have full visibility into their service accounts.
- 90% of IT leaders say properly managing NHIs is essential for a successful zero-trust implementation.
Questions worth separating out
Q: How should security teams govern AI agents that call internal tools from Kubernetes?
A: They should separate runtime containment from identity governance.
Q: Why do AI agents need identity and access governance if the model is already strong?
A: Because model strength does not control who or what the agent can touch in production.
Q: What breaks when teams rely on APM alone for AI applications?
A: APM alone breaks down when the system returns a technically successful response that is still wrong, unsafe, or unsupported.
Practitioner guidance
- Trace the internal agent workflow, not just the API edge Instrument retrieval, model calls, tool invocations, retries, and post-processing as separate spans so you can inspect the full execution path.
- Standardise GenAI span attributes across runtimes Use the same semantic fields for model name, token counts, operation type, and metadata across Python, TypeScript, Java, and Go.
- Route traces through a controlled export layer Use the OpenTelemetry Collector to batch, sample, redact, and forward spans to approved monitoring and evaluation backends.
What's in the full article
Braintrust's full guide covers the implementation detail this post intentionally leaves for the source:
- Runnable OpenTelemetry configuration for routing GenAI spans into Braintrust and other backends
- Framework-specific tracing examples for LangGraph, LangChain, OpenAI, and Anthropic instrumentation
- OTLP exporter settings, collector routing, and environment variable setup for production pipelines
- Evaluation recipe details for turning failing traces into reusable regression datasets
👉 Read Braintrust's guide to OpenTelemetry tracing for LLM and agent applications →
OpenTelemetry for LLM tracing: are your agent controls keeping up?
Explore further
19/08/2026 4:30 pm
OpenTelemetry solves visibility, not governance. The article correctly shows that tracing can expose model calls, tool paths, and token usage, but trace fidelity alone does not answer whether the agent had the right to act. For IAM and NHI programmes, the real issue is that runtime observability and delegated authority often live in separate control planes. Teams should treat tracing as evidence of behaviour, not evidence of approval.
A question worth separating out:
Q: How should organisations use data observability for AI reliability and audit readiness?
A: Monitor the data feeding AI systems with the same discipline used for critical reporting data. Track freshness, schema, volume, distribution, and lineage so that model inputs are continuously validated and any anomaly can be traced to an owner, an upstream cause, and a potentially affected business process.
👉 Read our full editorial: OpenTelemetry for LLM tracing exposes the gap in agent observability
