Join our Newsletter — 33% off our NHI Course

Why do AI agent tool calls drive 60-80% token waste?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 2 years ago
Posts: 20225
Topic starter  

TL;DR: 60 to 80% of tokens can be trapped in repeated payloads, which Archestra's blog post shows is why context layers matter for AI agent cost control. Agent workflows need tighter fetch, normalize, and serve rules so repeated tool results do not keep inflating prompts.

Editorial analysis by NHI Mgmt Group, based on content published by Archestra: “The context layer that pays for itself”.

By the numbers:

  • Most production traces show 60–80% of tokens going to tool results, not to reasoning.

Key questions

Q: Why do raw tool integrations make AI agents so expensive to run?

A: Because each tool call often returns a full object, then the model pays to ingest that object again when it reasons over it.

Q: What signals show that an agent context layer is actually working?

A: The clearest signs are fewer repeated upstream calls, smaller prompt payloads, and a lower share of tokens spent on tool results.

Q: How should teams stop AI agents from paying for the same data twice?

A: Use a context layer that caches repeated reads, normalises upstream payloads, and serves only the fields the agent actually needs.

Practitioner guidance

  • Define narrow read verbs for agent tools Expose only field-level or intent-specific reads where the task truly needs them, so the agent cannot default to full-record retrieval.
  • Cache normalised responses inside the trace Return a compact, stable representation for repeated reads within the same turn or workflow, and avoid rehydrating the full upstream object.
  • Measure repeated fetches before tuning models Track how often the same Jira issue, PR, or record is retrieved more than once in a single trace, then fix the retrieval pattern before changing models.

Bottom line: Agent cost overruns often come from repeated retrieval and oversized tool responses rather than from the model itself.

Glossary terms: AI Agent Context Graph, Scoped Read Verb, Prompt Bloat, Normalised Payload

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 1 week ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 20715
 

Context bloat is now a governance problem, not just a cost problem. When an agent repeatedly re-fetches the same operational record, the security issue is not merely duplicated spend. It is repeated exposure of the same business data across multiple reasoning steps, which expands the places where access, retention, and redaction must be controlled. Practitioners should treat context reduction as part of data minimisation, not just platform optimisation.

A few things that frame the scale:

  • 28% of secrets incidents now originate outside code repositories, in Slack, Jira, and Confluence, and are 13% more likely to be categorised as critical than code-based leaks, according to the State of Secrets Sprawl 2026.

A question worth separating out:

Q: Should security teams control agent access at the tool level or the field level?

A: Field-level control is stronger when the task only needs a narrow answer, because it prevents the agent from seeing or caching unnecessary data. Tool-level access is too coarse for many operational workflows. The better pattern is least privilege for both action and read scope, with the smallest useful payload by default.

👉 Read our full editorial: AI agent context layers cut token waste in tool-heavy workflows


This post was modified 1 week ago by NHI Mgmt Group

   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.