LLM token usage tracking: where do production controls fail?

TL;DR: Aggregate token totals hide the real causes of LLM cost and latency spikes, and teams need per-call, context-window, and per-span visibility to distinguish prompt bloat, context pressure, and runaway agent loops, according to Braintrust’s 2026 guide. The operational lesson is that token telemetry becomes a governance control only when it is attached to traces, prompts, and agent steps in production.

NHIMG editorial — based on content published by Braintrust: How to track LLM token usage (2026)

By the numbers:

• 64% of valid secrets leaked in 2022 are still valid and exploitable today, proving that detection alone is not enough without automated revocation.

Questions worth separating out

Q: What breaks when LLM token usage is only tracked as a monthly total?

A: Aggregate totals hide whether the problem is prompt bloat, context pressure, or a runaway agent loop.

Q: How should security teams govern AI agent token spend without losing accountability?

A: Treat token spend as an identity control problem.

Q: How do teams know when context-window pressure is becoming a production risk?

A: Watch p95 utilisation by route, tenant, model, and feature rather than relying on average usage.

Practitioner guidance

• Instrument every LLM call with usage telemetry Capture prompt tokens, completion tokens, cached-token fields, and reasoning-token fields on every call so production issues can be traced to the exact request rather than inferred from monthly totals.
• Track context-window utilisation by route and tenant Convert prompt_tokens into a percentage of model capacity, then alert on high p95 utilisation for specific tenants, features, or routes before overflow starts disrupting service.
• Break agent traces into inspectable spans Attach token counts, latency, retry count, and tool metadata to each span so oversized retrievals or runaway loops can be isolated without reading the full parent trace.

What's in the full article

Braintrust's full guide covers the operational detail this post intentionally leaves for the source:

• Exact SDK field names for OpenAI and Anthropic usage payloads, including cached-token and reasoning-token fields
• OpenTelemetry GenAI attribute examples for teams standardising on portable observability conventions
• Auto-instrumentation setup patterns for Python and TypeScript, including supported frameworks and streaming responses
• Dashboard and trace workflow details for rolling up token usage by prompt version, tenant, feature, and route

👉 Read Braintrust's guide to tracking LLM token usage in production →

LLM token usage tracking: where do production controls fail?

Explore further

View Full Forum →  |  NHI Foundation Course →