Topic starter
19/08/2026 3:45 pm
TL;DR: High token spend comes from three separate forces, inference waste, evaluation overhead, and incident exposure from unmonitored traces, according to Fiddler, while enterprises already spent $8.4 billion on LLM APIs in the first half of 2025. The real issue is attribution: without per-agent visibility, organisations cannot control cost, risk, or governance as agentic systems scale.
Editorial analysis by NHI Mgmt Group, based on content published by Fiddler: “How High Token Costs Quietly Erode AI ROI”.
By the numbers:
- Enterprise LLM API spending hit $8.4 billion in the first half of 2025.
Key questions
Q: How should teams reduce token waste without losing AI governance visibility?
A: Start by attributing spend to specific agents, workflows, and call types, then separate inference from evaluation and guardrail traffic.
Q: Why does agentic AI make token governance harder than standard LLM usage?
A: Agentic systems multiply calls across sub-agents, tools, retries, and retrieval steps, so token spend rises non-linearly and becomes harder to attribute.
Q: What do security teams get wrong about AI cost control?
A: They often treat cost as a finance-only issue and overlook the identity layer that drives usage.
Practitioner guidance
- Instrument token usage at the agent and span level Capture model name, input and output tokens, agent identifier, call type, and estimated cost for every request so spend can be attributed to a specific workflow.
- Separate inference, evaluation, and guardrail billing Track external evaluation calls as a distinct cost stream so oversight charges do not disappear into general model spend and create hidden budget pressure.
- Set spend and anomaly thresholds for agentic loops Alert on unusually fast token growth, repeated retries, and context expansion so cascading calls are caught before they consume a full budget cycle.
Bottom line: Token spend in agentic AI is a governance signal as much as a financial one, because waste, evaluation overhead, and blind spots compound together.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
19/08/2026 4:23 pm
Token governance is becoming a control plane problem, not a procurement problem. Once organisations deploy agents at scale, token spend reflects routing quality, evaluation design, and trace visibility rather than model price alone. Finance teams can see the bill, but only governance controls can explain which agent, workflow, or policy decision created it. Practitioners should treat token attribution as a core control requirement, not an optional optimisation layer.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
- A Harris Poll survey of more than 300 technology decision-makers found that 86% expect agentic AI to deliver positive ROI, yet fewer than half had AI governance policies in place.
A question worth separating out:
Q: What happens when organisations down-sample AI traces to save money?
A: They create blind spots where hallucinations, policy violations, and unsafe agent actions can pass unnoticed. Down-sampling may reduce billable evaluation traffic, but it also weakens evidentiary coverage and makes incident reconstruction harder. In regulated use cases, that trade-off can create far greater cost than the savings it produces.
👉 Read our full editorial: Token governance is now central to AI ROI and agentic scale