Notifications
Clear all
Topic starter
19/08/2026 3:26 pm
TL;DR: LLM traces capture the full request path across retrieval, tool calls, prompts, and model spans, while logs usually only preserve the final output, making root-cause analysis, latency analysis, and regression evaluation much harder, according to Braintrust. For identity and AI governance teams, the shift matters because production traces become the evidence layer for understanding how delegated steps, data inputs, and system decisions shaped an answer.
NHIMG editorial — based on content published by Braintrust: LLM tracing: The complete guide
By the numbers:
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes.
- 97% of NHIs carry excessive privileges, increasing unauthorised access and broadening the attack surface.
- Only 5.7% of organisations have full visibility into their service accounts.
Questions worth separating out
Q: How should security teams use LLM tracing in AI governance programmes?
A: Security teams should use LLM tracing as evidence for how an AI response was produced, not just whether it succeeded.
Q: Why do AI applications need tracing when logs already exist?
A: AI applications need tracing because logs usually capture only the final event, while the real failure may sit in retrieval, tool data, or prompt assembly.
Q: What should teams measure to know whether LLM tracing is working?
A: Teams should measure whether traces let them identify the step that introduced bad context, whether they can compare runs span by span, and whether failed traces reliably become regression cases.
Practitioner guidance
- Trace every retrieval and tool hop Instrument the full request path so retrieval results, tool inputs and outputs, retries, and model calls are captured in one trace.
- Correlate AI traces with service identity usage Map traces to the service accounts, tokens, or API keys used by retrieval and tool connectors so privileged AI actions can be reviewed alongside the request that triggered them.
- Turn failed traces into regression cases Promote traces that produced incorrect or unsafe outputs into evaluation datasets, then rerun them after prompt, tool, or model changes to see whether the failure mode still exists.
What's in the full article
Braintrust's full guide covers the implementation detail this post intentionally leaves at the analytical level:
- Step-by-step setup for auto-instrumentation, manual spans, and SDK wrapping across common AI stacks
- OpenTelemetry integration patterns for teams that already emit GenAI telemetry into an existing observability pipeline
- Framework-specific tracing examples for agent runtimes such as LangGraph, CrewAI, and the Vercel AI SDK
- Production scoring and regression workflows that turn traces into reusable evaluation cases
👉 Read Braintrust's complete guide to LLM tracing and request-path debugging →
LLM tracing vs logging: what security and AI teams need to know?
Explore further
19/08/2026 4:10 pm
LLM tracing is now part of governance, not just debugging. The moment a model call depends on retrieval, tool execution, and delegated actions, the audit question changes from what was returned to how it was produced. That makes trace data relevant to AI governance, access control, and incident review, because a response can only be trusted if the underlying execution path is inspectable. Practitioners should treat trace coverage as a control boundary, not an engineering luxury.
A question worth separating out:
Q: How can organisations connect AI tracing to NHI governance?
A: Organisations should connect traces to the service identities used by retrieval systems, tool APIs, and agent workflows. That lets teams review which credentials were exercised, which systems were touched, and whether delegated access stayed inside its intended boundary. In practice, tracing and NHI governance should share the same evidence trail.
👉 Read our full editorial: LLM tracing exposes the execution path behind bad AI answers
