Notifications
Clear all
Topic starter
20/08/2026 6:00 pm
TL;DR: Aggregate token totals hide the real causes of LLM cost and latency spikes, and teams need per-call, context-window, and per-span visibility to distinguish prompt bloat, context pressure, and runaway agent loops, according to Braintrust’s 2026 guide. The operational lesson is that token telemetry becomes a governance control only when it is attached to traces, prompts, and agent steps in production.
NHIMG editorial — based on content published by Braintrust: How to track LLM token usage (2026)
By the numbers:
- 64% of valid secrets leaked in 2022 are still valid and exploitable today, proving that detection alone is not enough without automated revocation.
Questions worth separating out
Q: What breaks when LLM token usage is only tracked as a monthly total?
A: Aggregate totals hide whether the problem is prompt bloat, context pressure, or a runaway agent loop.
Q: How should security teams govern AI agent token spend without losing accountability?
A: Treat token spend as an identity control problem.
Q: How do teams know when context-window pressure is becoming a production risk?
A: Watch p95 utilisation by route, tenant, model, and feature rather than relying on average usage.
Practitioner guidance
- Instrument every LLM call with usage telemetry Capture prompt tokens, completion tokens, cached-token fields, and reasoning-token fields on every call so production issues can be traced to the exact request rather than inferred from monthly totals.
- Track context-window utilisation by route and tenant Convert prompt_tokens into a percentage of model capacity, then alert on high p95 utilisation for specific tenants, features, or routes before overflow starts disrupting service.
- Break agent traces into inspectable spans Attach token counts, latency, retry count, and tool metadata to each span so oversized retrievals or runaway loops can be isolated without reading the full parent trace.
What's in the full article
Braintrust's full guide covers the operational detail this post intentionally leaves for the source:
- Exact SDK field names for OpenAI and Anthropic usage payloads, including cached-token and reasoning-token fields
- OpenTelemetry GenAI attribute examples for teams standardising on portable observability conventions
- Auto-instrumentation setup patterns for Python and TypeScript, including supported frameworks and streaming responses
- Dashboard and trace workflow details for rolling up token usage by prompt version, tenant, feature, and route
👉 Read Braintrust's guide to tracking LLM token usage in production →
LLM token usage tracking: where do production controls fail?
Explore further
20/08/2026 6:38 pm
Token observability is becoming a governance control, not just a FinOps metric. The article shows why a single daily or monthly total is too coarse for production AI systems. When the same spike can be caused by prompt bloat, context pressure, or an agent loop, leaders need telemetry that supports accountability at the call and span level. For AI governance teams, the practical conclusion is that token data belongs in the control plane, not only in billing dashboards.
A question worth separating out:
Q: Why do agent loops create token problems that normal dashboards miss?
A: Because the expensive behaviour often happens inside a nested span, not at the top level. A retrying tool call, oversized retrieval, or sub-agent loop can consume most of the budget while the parent trace still looks ordinary. Span-level token attribution is what reveals the real cost driver.
👉 Read our full editorial: LLM token tracking reveals where production behavior breaks down
