Topic starter
20/08/2026 5:59 pm
TL;DR: LLM cost tracking breaks when provider invoices are treated as the source of truth, because they cannot explain which user, feature, prompt change, or agent run caused spend to rise, according to Braintrust. Request-level metadata, cost rollups, and kill switches make LLM economics governable before runaway loops and retry storms turn into budget and reliability problems.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “How to track LLM costs (2026): A playbook for per-user, per-feature, and per-agent-run attribution”.
Key questions
Q: What breaks when LLM cost is tracked only in invoices?
A: Finance may see the spend, but engineering cannot identify which user, feature, prompt change, or agent run caused it.
Q: Why do request-level metadata and cost rollups matter for AI spend control?
A: They turn spend into an operational signal.
Q: How do teams know if LLM cost controls are actually working?
A: Look for stable cost per feature request, bounded cost per agent run, and no widening gap between median and p99 spend.
Practitioner guidance
- Implement request-time metadata tagging Attach user_id, customer_id, feature, deployment, prompt_version, and agent_run_id at the model call so cost can be rolled up from the same trace that records the request.
- Build cost rollups by operational dimension Create daily rollups for cost per user, cost per feature request, cost per agent run, and cost per successful eval so spend can be evaluated against product and quality outcomes.
- Set low-cardinality alerts and high-cardinality drill-downs Alert on feature, deployment, and model combinations, then use user and agent run identifiers only for trace-level investigation when an alert fires.
Bottom line: LLM spend becomes difficult to govern when the organisation treats finance invoices as the primary record instead of request-scoped telemetry.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
20/08/2026 6:38 pm
LLM cost attribution is becoming a governance problem, not just a FinOps problem. When model spend cannot be tied to a user, feature, prompt version, or agent run, the organisation has lost operational accountability for AI usage. That creates the same kind of control gap identity teams see when shared credentials obscure who actually performed an action. The practitioner conclusion is that request metadata must be treated as part of the control plane, not as optional analytics.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: What should security and AI teams do when agent runs start retrying or looping?
A: Treat the behaviour as a control failure, not just a usage anomaly. Stop the run, inspect the span tree for repeated tool calls or retry storms, and tighten the budget, ceiling, or routing rule that allowed the loop to continue without intervention.
👉 Read our full editorial: LLM cost attribution needs request-level metadata, not invoices