Why do AI bills rise even when token prices fall?

Why This Matters for Security Teams

AI bills usually rise because the workload expands faster than the unit price falls. More context is packed into each prompt, more model turns are needed to finish a task, and more premium models are routed in when accuracy matters. For security teams, that means cost is increasingly driven by orchestration, not just tokens. The same pattern appears in secret-heavy workflows, where complexity and sprawl dominate the budget, as shown in The State of Secrets in AppSec.

The practical risk is misreading cheaper tokens as a cost-control win when the underlying workflow is becoming more autonomous and more expensive to govern. When agents can call tools, retry actions, and chain subtasks, one user request can become a sequence of billable actions, secret lookups, and policy checks. NIST’s NIST SP 800-63 Digital Identity Guidelines are useful here because they reinforce that identity assurance matters when a system is acting on behalf of someone, not just consuming API volume. In practice, many security teams discover runaway AI spend only after a workflow has already been promoted into production.

How It Works in Practice

The unit cost of AI is rarely the model price alone. A task may start with retrieval over large context windows, then move through multiple model calls for planning, validation, summarisation, and exception handling. If the workflow is agentic, the system may also issue tool calls, fetch secrets, open tickets, invoke external APIs, and re-query the model after each action. Every step adds latency, governance overhead, and cost. That is why billing should be measured per completed task, per successful outcome, or per business transaction, not only per token consumed.

Security teams also need to track where cost inflation hides. A cheaper base model can still be more expensive overall if it requires more retries, more human review, or a premium router that sends sensitive prompts to a larger model. This is especially true when secrets are involved. NHIMG research shows that secret exposure is already a material operational drag in AI-heavy environments, including the Guide to the Secret Sprawl Challenge and the Vercel Context.ai OAuth Supply Chain Breach. If each autonomous action requires a fresh token, credential check, or policy evaluation, the cost model becomes a chain of short-lived decisions rather than a simple token ledger.

• Measure cost by completed task, not prompt volume alone.
• Separate base model spend from orchestration, retrieval, tool, and review costs.
• Track retries, escalations, and premium routing as first-class cost drivers.
• Use short-lived access and least privilege so agent actions do not create hidden cost and risk spillover.

These controls tend to break down in high-variance agent workflows because the number of model calls and tool invocations changes with every task.

Common Variations and Edge Cases

Tighter cost controls often increase operational overhead, requiring organisations to balance spend visibility against workflow flexibility. Some teams can cap costs with strict context limits, but that can reduce answer quality or force more human intervention, which simply shifts the cost elsewhere.

There is no universal standard for this yet, but current guidance suggests treating agent spend as a governance problem as much as a FinOps problem. For example, a workflow that uses retrieval-augmented generation may look efficient at the token layer while still becoming expensive because it repeatedly queries internal data, rehydrates context, and re-checks permissions. That same pattern can amplify exposure if credentials or API keys are embedded in prompts or tool configs, which is why NHIMG’s research on the State of Secrets Sprawl 2026 is directly relevant to AI operations.

The edge case is autonomous escalation. When an agent is allowed to decide when to retry, when to call a more capable model, or when to invoke a third-party tool, the bill can spike even if the original prompt is small. Teams that only optimise token price often miss that the expensive part is the chain reaction. In practice, many organisations find the problem only after a high-volume agent has already been allowed to run unchecked across production tools.

Related resources from NHI Mgmt Group

• When does just-in-time access reduce risk for agentic AI, and when does it fall short?
• How does the rise of AI identities impact traditional IAM systems?
• When is it crucial to implement least-privilege access for AI agents?
• What is the difference between managed identities and hardcoded secrets for AI agents?