Notifications
Clear all
Topic starter
20/08/2026 5:59 pm
TL;DR: LLM cost tracking breaks when provider invoices are treated as the source of truth, because they cannot explain which user, feature, prompt change, or agent run caused spend to rise, according to Braintrust. Request-level metadata, cost rollups, and kill switches make LLM economics governable before runaway loops and retry storms turn into budget and reliability problems.
NHIMG editorial — based on content published by Braintrust: How to track LLM costs (2026): A playbook for per-user, per-feature, and per-agent-run attribution
Questions worth separating out
Q: How should teams attribute LLM cost at the request level?
A: Tag each model call with request metadata at creation time, including user, customer, feature, deployment, prompt version, and agent run identifiers.
Q: Why do agentic workflows make LLM cost harder to control?
A: Agentic workflows can trigger repeated tool calls, retries, and long execution chains inside one run, which turns a small failure into a cost spike.
Q: What breaks when LLM cost is tracked only in invoices?
A: Finance may see the spend, but engineering cannot identify which user, feature, prompt change, or agent run caused it.
Practitioner guidance
- Implement request-time metadata tagging Attach user_id, feature, deployment, prompt_version, agent_run_id, and customer_id at the model call site so every trace can support cost rollups and investigation.
- Build cost rollups for operational decisions Create cost per user, cost per feature request, cost per agent run, and cost per customer reports, then use them to identify retry storms, runaway loops, and margin pressure.
- Set run-level ceilings for autonomous workflows Define token, tool-call, retry, and span-depth limits inside the agent framework so a misbehaving run can be stopped before it finishes its loop.
What's in the full article
Braintrust's full guide covers the operational detail this post intentionally leaves for the source:
- Step-by-step span tagging patterns for OpenAI client wrappers in TypeScript and Python
- Token-field conventions for cached tokens, cache writes, and custom model cost logging
- Practical rollup examples for user, feature, customer, and agent-run cost analysis
- Release-gate logic that combines cost, eval pass rate, and latency before production rollout
👉 Read Braintrust's guide to tracking LLM costs with request-level attribution →
LLM cost attribution and agent-run visibility: what teams miss?
Explore further
20/08/2026 6:38 pm
Request-level attribution is the real control boundary for AI spend. Invoice-only visibility is too coarse for operational governance because it collapses many behaviours into one number. When user, feature, prompt, and agent-run metadata are attached at call time, cost becomes auditable in the same way other production events are auditable. That shift matters for security and platform teams because it turns AI usage into governed runtime activity, not an anonymous expense line. Practitioners should treat trace-level attribution as the minimum viable control for AI economics.
A question worth separating out:
Q: How do teams stop a misbehaving agent run from wasting budget?
A: Use a kill switch in the agent framework that watches token count, tool-call count, retry count, and span depth. When one of those ceilings is crossed, stop the run before it completes the next loop. That containment belongs in the execution path, because post-hoc review arrives too late to prevent the cost.
👉 Read our full editorial: LLM cost attribution needs request-level metadata, not invoices
