Notifications
Clear all
Topic starter
21/08/2026 4:50 pm
TL;DR: LLM costs often rise because total-spend dashboards hide the prompts, tool calls, retries, and model choices driving spend, and Braintrust says span-level tracing, prompt experimentation, model comparison, and evals can expose and reduce that waste without degrading output quality. The key governance shift is that cost optimisation becomes an evidence-based release process, not a model-swap guess.
NHIMG editorial — based on content published by Braintrust: How to reduce costs for LLMs using Braintrust
By the numbers:
- Braintrust says AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers.
- Braintrust says 28% of secrets incidents now originate outside code repositories, in Slack, Jira, and Confluence, and are 13% more likely to be categorised as critical than code-based leaks.
- Braintrust says 64% of valid secrets leaked in 2022 are still valid and exploitable today, showing that detection alone does not resolve exposure.
Questions worth separating out
Q: How should teams reduce LLM costs without breaking production quality?
A: Start with request-level tracing so you know which prompts, models, retrieval steps, or tool calls drive cost.
Q: Why do agentic AI systems often cost more than teams expect?
A: Because the visible user request usually hides many internal steps.
Q: What signals show that an LLM workflow is becoming inefficient?
A: Look for growing system prompts, retrieval steps that return oversized context, repeated tool calls, and traces where one child span is far more expensive than its siblings.
Practitioner guidance
- Implement span-level cost attribution Attach estimated cost and token counts to every request span so product, engineering, and security teams can identify the exact prompt, retrieval step, or tool call driving spend.
- Inspect agent loops and retrieval scope Review trace trees for repeated tool calls, oversized context windows, and retrieval steps that pull more data than the task requires, then narrow the workflow before switching models.
- Run prompt and model experiments on production traces Compare candidate prompts and models against the same real inputs, using quality scores to verify that a cheaper option still meets output standards.
What's in the full article
Braintrust's full article covers the operational detail this post intentionally leaves for the source:
- Side-by-side examples of trace views, timeline views, and child-span drill-downs for identifying high-cost LLM steps
- Concrete prompt editing patterns that reduce token usage without changing the business logic of the workflow
- Model comparison examples showing cost and quality scores across different model classes
- CI/CD eval workflow details, including how to block merges when quality thresholds are not met
👉 Read Braintrust's guide to reducing LLM costs with tracing and evals →
LLM tracing for cost control: what changes for teams?
Explore further
21/08/2026 5:09 pm
Span-level observability is now a governance requirement for production AI. Total-cost dashboards can tell teams that spending is increasing, but they cannot identify the prompt, tool call, or model decision causing the rise. That gap is especially problematic in agentic workflows where one request can cascade through many steps. The practical conclusion is that AI cost control belongs alongside runtime governance, not beside it.
A question worth separating out:
Q: How do organisations govern cost optimisation in production AI safely?
A: Use CI/CD evals as the approval gate for prompt and model changes, and make expensive production traces reusable test cases. That gives the team a controlled way to improve spend while preserving quality and creates a durable record of known failure patterns.
👉 Read our full editorial: Braintrust shows how span-level tracing cuts LLM costs
