Token Dilemma

The token dilemma is the practical limit on how broadly an organisation can apply expensive AI analysis. It describes the trade-off between deeper reasoning and the cost of running that reasoning across large codebases, where routing decisions matter more than model price alone.

Expanded Definition

The token dilemma describes the operational limit at which organisations stop being able to run expensive AI analysis everywhere and must choose where deeper reasoning is justified. In NHI security and agentic AI operations, the constraint is not only model price; it is the cost of routing, context assembly, and repeated inference across large codebases, ticket streams, and identity logs.

Definitions vary across vendors, but the practical meaning is consistent: not every event deserves the same model depth. A triage layer may use lightweight classification first, then escalate only high-risk items to a stronger model. That approach aligns with the risk-based logic reflected in the NIST Cybersecurity Framework 2.0, where the control objective is to focus protection effort where business impact is highest.

For NHIs, the dilemma is especially sharp because token-heavy workflows often touch secrets, service accounts, and agent tool calls at machine speed. The most common misapplication is treating every code diff, alert, or workflow step as equally worthy of deep reasoning, which occurs when routing rules are absent and teams rely on a single expensive model for all decisions.

Examples and Use Cases

Implementing the token dilemma rigorously often introduces latency and policy complexity, requiring organisations to weigh broader inspection coverage against the cost of slower pipelines and heavier orchestration.

• A security team uses a small classifier to detect obvious secret patterns, then sends only ambiguous findings to a larger model for context-aware verification.
• An agent platform limits deep analysis to privileged actions, while routine read-only tasks are handled with cheaper reasoning paths and cached context.
• Code review tooling escalates only files that touch authentication, token storage, or webhook handling, reducing unnecessary token consumption across the full repository.
• Teams studying the Guide to the Secret Sprawl Challenge use routing to prioritise likely exposure points rather than scanning every artifact with the same depth.
• Investigation workflows for incidents like the Salesloft OAuth token breach focus deeper reasoning on identity-linked events where token misuse would have immediate blast-radius implications.

In practice, the idea also maps to modern AI operations guidance from the NIST Cybersecurity Framework 2.0 because controls should be applied proportionally to risk, not uniformly to every signal.

Why It Matters in NHI Security

Token-aware routing matters because expensive analysis cannot be the default response to every secret, credential, or agent event. NHI environments already generate high-volume exposure paths, and the wrong routing decision can leave genuine compromise indicators buried under low-value noise. That is why NHIMG research on the State of Secrets Sprawl 2026 is so relevant: AI-related credential leaks surged 81.5% year-over-year in 2025, showing how quickly identity exposure expands when governance does not keep pace.

The operational risk is not only budget burn. Over-analysis can slow detection, while under-analysis lets exposed tokens persist, especially in agentic workflows where tool access and execution authority are tightly coupled. The 2025 State of NHIs and Secrets in Cybersecurity reports that 44% of NHI tokens are exposed in the wild, which means routing must prioritize likely compromise paths, not just volume.

Organisations typically encounter the true cost of the token dilemma only after a secrets incident, at which point prioritisation, escalation, and automated revocation become operationally unavoidable to address.

Related resources from NHI Mgmt Group

• Why is OAuth token management critical in cloud environments?
• How should organizations respond to OAuth token abuse incidents?
• How should teams respond when a service account token is exposed?
• What is the difference between token theft and privilege escalation in managed identity attacks?