AI token spend by team: what controls actually hold budgets?

TL;DR: Most engineering teams can see total AI spend but not decompose it by team, model, or person, making token counts a poor proxy for cost when caching, model mix, and passthrough subscriptions distort the bill, according to Tetrate. The operational problem is governance, not raw usage: cost control depends on identity, attribution, and budget enforcement at the gateway.

Editorial analysis by NHI Mgmt Group, based on content published by Tetrate: “Token Optimization: How Engineering Leaders Control AI Token Costs”.

By the numbers:

• Organization scope, last 30 days: 42B tokens, of which 37.6B cache reads, 2.2B cache creation, 2B fresh input, and 228.7M output.

Key questions

Q: How should teams govern AI consumption when spend is spread across multiple tools?

A: Start by assigning one control owner for AI consumption governance and require shared evidence from finance, IT, and security.

Q: Why do AI token totals and actual cost often diverge?

A: Because model pricing is not linear.

Q: What breaks when AI usage is not mapped to the right team?

A: Chargeback and budget ownership break first, followed by any attempt to compare spend across departments.

Practitioner guidance

• Implement gateway-level cost attribution Capture caller identity, model, token counts, and team mapping in the same control plane so spend can be decomposed consistently across providers.
• Reconcile identity-provider team mappings Audit every No team record and every mismatched user-to-team assignment before using usage data for chargeback or budget enforcement.
• Review effective cost per million tokens Compare token volume, cache reuse ratio, and model mix together so you can spot expensive workloads that a simple token cap would miss.

Bottom line: AI token spend is hard to control when usage data cannot be tied back to teams, people, and models.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →