Token governance is now central to AI ROI and agentic scale

At a glance

What this is: Fiddler argues that token governance is now a core AI ROI issue because waste, evaluation overhead, and unmonitored trace exposure compound as agentic systems scale.

Why it matters: This matters to IAM and AI governance teams because attribution, policy enforcement, and observability increasingly determine whether AI agents remain controllable, auditable, and financially viable.

By the numbers:

• Enterprise LLM API spending hit $8.4 billion in the first half of 2025.

Context

Token governance is the discipline of linking AI usage, cost, and policy accountability to specific agents, workflows, and traces. In AI systems, token spend is not just a billing issue because it often reflects waste, hidden evaluation traffic, and control blind spots that finance teams alone cannot fix.

Fiddler's analysis frames the problem as a governance gap rather than a pricing problem. For identity and AI security teams, the key issue is whether the organisation can attribute action, cost, and policy violations to a specific agent or workflow before scale turns those blind spots into operational and compliance risk.

Key questions

Q: How should teams reduce token waste without losing AI governance visibility?

A: Start by attributing spend to specific agents, workflows, and call types, then separate inference from evaluation and guardrail traffic. That lets teams identify whether waste comes from prompt design, model choice, retrieval volume, or oversight overhead. If controls only show a daily total, they cannot support meaningful governance or optimisation.

Q: Why does agentic AI make token governance harder than standard LLM usage?

A: Agentic systems multiply calls across sub-agents, tools, retries, and retrieval steps, so token spend rises non-linearly and becomes harder to attribute. A single user request can create several hidden cost paths, and each path may also create its own oversight burden. Governance has to move from billing review to per-action accountability.

Q: What do security teams get wrong about AI cost control?

A: They often treat cost as a finance-only issue and overlook the identity layer that drives usage. Without attribution, shadow AI discovery, and policy enforcement at the request path, teams can reduce waste in one area while leaving the real source of token growth untouched.

Q: What happens when organisations down-sample AI traces to save money?

A: They create blind spots where hallucinations, policy violations, and unsafe agent actions can pass unnoticed. Down-sampling may reduce billable evaluation traffic, but it also weakens evidentiary coverage and makes incident reconstruction harder. In regulated use cases, that trade-off can create far greater cost than the savings it produces.

Technical breakdown

Why inference waste becomes a governance issue in agentic AI

Inference waste appears when systems spend tokens on avoidable work such as oversized prompts, overly large context windows, repeated calls without caching, or using an expensive model for a simple task. In agentic systems, that waste compounds because one user request can fan out across multiple agents and tool calls, multiplying token consumption across the workflow. The architectural issue is not only cost inefficiency. It is that uncontrolled prompt design and routing logic can hide where value is actually created or lost.

Practical implication: tie token telemetry to each agent and workflow so teams can identify which design choices are inflating spend.

Evaluation overhead and the hidden cost of external LLM judging

Evaluation overhead arises when AI outputs are checked by another LLM, especially when each trace triggers one or more external calls. That creates a second bill alongside inference, often called the evaluation trust tax. Because the cost scales with trace volume and metric count, teams can reduce model spend in one area while increasing total AI operating cost elsewhere. The technical problem is that evaluation becomes a separate consumption layer, not a free control, when it sits outside the production environment.

Practical implication: separate evaluation telemetry from inference telemetry so the organisation can see whether governance checks are creating a second cost centre.

Why unmonitored traces create both cost and incident exposure

When teams down-sample traces to control evaluation cost, they create an unmonitored residue of agent activity. That residue is where hallucinations, policy violations, and unsafe actions can escape detection, especially in regulated workflows or customer-facing interactions. The control failure is not just incomplete sampling. It is the loss of evidence needed to prove what an agent did, why it did it, and whether a policy was breached. In governance terms, the organisation stops being able to attribute risk to a specific session or action path.

Practical implication: keep trace coverage high enough that oversight remains evidentiary, not merely statistical.

Threat narrative

Attacker objective: The practical objective is not theft but unchecked cost growth and reduced oversight, which together undermine AI ROI and governance confidence.

1. Entry occurs through high-volume agentic usage, where repeated prompts, retrieval calls, and tool invocations generate token consumption across sessions and workflows.
2. Escalation happens when evaluation traffic and retry loops multiply the cost surface, turning routine monitoring into a separate and often hidden spend stream.
3. Impact appears when teams down-sample traces to stay within budget, leaving hallucinations, compliance violations, and unsafe agent actions unobserved.

Breaches seen in the wild

• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Token governance is becoming a control plane problem, not a procurement problem. Once organisations deploy agents at scale, token spend reflects routing quality, evaluation design, and trace visibility rather than model price alone. Finance teams can see the bill, but only governance controls can explain which agent, workflow, or policy decision created it. Practitioners should treat token attribution as a core control requirement, not an optional optimisation layer.

Observability is now the point where AI governance and identity governance meet. AI systems that cannot be tied to a specific agent, workflow, or session are difficult to govern because responsibility becomes diffuse. That makes per-agent telemetry analogous to identity traceability in IAM: if you cannot identify the actor, you cannot reliably assign accountability, enforce policy, or reconstruct behaviour after the fact. Practitioners should align AI observability with policy enforcement and audit trails.

Evaluation overhead creates governance debt when oversight is externally billed. A control that only works by generating expensive external calls is vulnerable to being weakened under budget pressure. This creates a named pattern: trace coverage erosion, where cost pressure causes teams to monitor less precisely and accept higher blind spots. The result is a weaker assurance model, not just lower spend. Practitioners should design oversight that preserves coverage without making governance itself a cost driver.

Agentic scale changes the economics of control enforcement. Multi-agent systems, retries, and retrieval-heavy workflows can expand cost faster than teams expect because every new path increases both inference and monitoring load. The market signal is that governance for agentic systems is moving toward continuous attribution, policy-aware telemetry, and cost-aware assurance. Practitioners should expect AI control frameworks to converge with identity-style accountability models.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• A Harris Poll survey of more than 300 technology decision-makers found that 86% expect agentic AI to deliver positive ROI, yet fewer than half had AI governance policies in place.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

Trace coverage erosion: This article shows why cost pressure can quietly weaken AI oversight by pushing teams toward lower sampling and narrower evaluation. Once that happens, governance becomes statistical instead of evidentiary, which is a poor fit for regulated or customer-facing agentic systems.

Token governance will increasingly sit alongside identity-style accountability in AI programmes because the same control problem recurs: if you cannot attribute action to a specific actor or workflow, you cannot govern it well. Teams should expect observability, policy enforcement, and cost controls to converge in the same operating model.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That broader leakage pattern reinforces the same operational lesson here: governance fails fastest where scale outpaces attribution.

For practitioners

• Instrument token usage at the agent and span level Capture model name, input and output tokens, agent identifier, call type, and estimated cost for every request so spend can be attributed to a specific workflow.
• Separate inference, evaluation, and guardrail billing Track external evaluation calls as a distinct cost stream so oversight charges do not disappear into general model spend and create hidden budget pressure.
• Set spend and anomaly thresholds for agentic loops Alert on unusually fast token growth, repeated retries, and context expansion so cascading calls are caught before they consume a full budget cycle.
• Keep trace coverage high for regulated workflows Avoid heavy down-sampling where outputs can affect customers, compliance decisions, or financial advice, because unmonitored traces are where incidents hide.
• Map policy violations to specific agents and sessions Require every enforcement action to retain the originating agent, workflow, and policy context so remediation can target the actual source of waste or risk.

Key takeaways

• Token spend in agentic AI is a governance signal as much as a financial one, because waste, evaluation overhead, and blind spots compound together.
• Per-agent attribution matters because aggregate dashboards hide which workflows are driving cost, compliance exposure, and runaway evaluation traffic.
• Organisations that preserve trace coverage and separate oversight cost from inference cost are better positioned to control both ROI and accountability.

Key terms

• Token governance: Token governance is the control of credential lifecycle, scope, and runtime use for non-human identities. It combines inventory, issuance policy, behavioral monitoring, and revocation so that access remains limited, visible, and accountable across dynamic environments.
• Evaluation Trust Tax: The hidden cost of proving AI quality when verification depends on external model calls or repeated assessment passes. It matters because assurance can erase savings if the organisation pays too much to validate the outputs it has already generated.
• Trace Coverage: Trace coverage is the proportion of AI actions, sessions, or outputs that are captured for review and analysis. High coverage supports evidence, accountability, and incident reconstruction, while low coverage turns governance into sampling and leaves material behaviour outside oversight.
• Per-Agent Attribution: Per-agent attribution is the ability to tie cost, behaviour, and policy decisions to a specific AI agent or workflow step. It is essential when multiple agents, tools, and retries are involved because aggregate dashboards hide which actor created the spend or the risk.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and agentic AI identity in practical terms. It helps security and identity teams build the accountability model that production AI now requires.