Notifications
Clear all
Topic starter
19/08/2026 3:51 pm
TL;DR: Python LLM apps often break in retrieval, tool calls, retries, async tasks, and streaming paths that plain logs cannot reconstruct, according to Braintrust. The practical shift is from disconnected debugging to span-based traces that preserve inputs, outputs, latency, metadata, and replay context for release checks.
NHIMG editorial — based on content published by Braintrust: How to trace LLM apps in Python (2026)
Questions worth separating out
Q: How should teams trace Python LLM apps across tools, retries, and async tasks?
A: Trace the application at each meaningful boundary, not only at the final model response.
Q: Why do LLM traces need metadata beyond prompts and outputs?
A: Prompts and outputs show only part of the system’s behaviour.
Q: What breaks when span context is lost in Python agent workflows?
A: When span context drops, nested calls stop lining up with the parent request and the trace no longer proves which action belonged to which run.
Practitioner guidance
- Instrument every workflow boundary Capture retrieval, model calls, tool execution, retries, and post-processing as named spans so the full request path remains inspectable across production traffic.
- Preserve metadata at request entry Attach user ID, org ID, session ID, prompt version, and environment context at the first span so traces can be filtered for investigations and evaluation datasets.
- Validate async and thread propagation Test worker pools, background tasks, and streaming paths to confirm span context survives beyond the request handler and that traces remain complete under concurrency.
What's in the full article
Braintrust's full guide covers the operational detail this post intentionally leaves for the source:
- Step-by-step Python examples for auto-instrumentation, manual spans, and OpenTelemetry export across supported libraries.
- Framework-specific tracing patterns for LangChain, LangGraph, LlamaIndex, and FastAPI in production services.
- Concurrency handling details for async execution, streaming responses, and traced thread pools.
- Dataset and eval workflow examples that show how production traces become release checks.
👉 Read Braintrust's guide to tracing Python LLM apps in production →
Python LLM tracing: what it means for agent debugging and evals?
Explore further
19/08/2026 4:27 pm
LLM tracing is now a governance control, not just an engineering convenience. Once a Python AI workflow spans retrieval, tools, retries, and streaming, plain logs no longer provide a reliable record of what the system actually did. That makes span integrity a control issue for security, compliance, and release management. Teams that cannot reconstruct execution paths cannot reliably review agent behaviour, credential use, or failure propagation.
A question worth separating out:
Q: How do teams decide when tracing is good enough for release governance?
A: Tracing is good enough when a production run can be reconstructed end to end, filtered by request metadata, and reused as a stable eval case. Teams should verify that provider calls, custom application spans, and concurrency paths all land in the same trace workflow. If any of those links are missing, release governance is still blind.
👉 Read our full editorial: Python LLM tracing exposes the observability gap in agent workflows
