Topic starter
21/08/2026 4:50 pm
TL;DR: LLM costs often rise because total-spend dashboards hide the prompts, tool calls, retries, and model choices driving spend, and Braintrust says span-level tracing, prompt experimentation, model comparison, and evals can expose and reduce that waste without degrading output quality. The key governance shift is that cost optimisation becomes an evidence-based release process, not a model-swap guess.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “How to reduce costs for LLMs using Braintrust”.
Key questions
Q: How should teams reduce LLM costs without breaking production quality?
A: Start with request-level tracing so you know which prompts, models, retrieval steps, or tool calls drive cost.
Q: Why do LLM costs often stay hidden until production?
A: Testing traffic is usually too small and too clean to expose context bloat, retry loops, or inefficient tool use.
Q: What breaks when teams rely only on cloud billing dashboards to manage LLM spend?
A: Cloud billing shows totals, but it does not explain which prompt, agent, model, or retry pattern drove the increase.
Practitioner guidance
- Map cost to individual spans Attach estimated cost and token counts to each model call, tool call, and retry path so the expensive step is visible inside the trace rather than hidden in aggregate spend.
- Investigate the heaviest branches first Start with the most expensive production traces and drill into child spans to identify whether prompt bloat, retrieval overreach, or repeated tool calls is driving the bill.
- Trim prompts with real workload evidence Reduce redundant instructions, oversized examples, and system prompt growth only after comparing prompt variants against the same traced inputs.
Bottom line: LLM cost problems are usually caused by hidden execution details, not just by the choice of model.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
21/08/2026 5:09 pm
Span-level observability is the real cost-control boundary for production LLM systems. Once prompts, retrieval, tools, and retries are all part of the same request path, total spend alone becomes too coarse to govern. Cost control moves from budgeting after the fact to understanding which execution path created the bill, which is the only way to make optimisation repeatable. Practitioners should treat trace visibility as a control requirement, not a reporting convenience.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should teams switch to cheaper models or fix prompts first?
A: Usually fix prompts and workflow structure first, because oversized prompts, retrieval bloat, and retry loops often account for a large share of cost. Switch models when tracing shows the prompt is already tight and the remaining spend comes from a model that is unnecessarily strong for the task.
👉 Read our full editorial: Braintrust shows how span-level tracing cuts LLM costs