Why do AI systems with multiple tool calls become slow so quickly?

Each extra call adds its own network, context, model, and post-processing overhead, so small delays compound into seconds. Agentic systems are especially exposed because one user request can trigger several sequential decisions. The fix is to trace the full execution path, collapse redundant steps, and budget latency per stage rather than per request.

Why This Matters for Security Teams

Tool-rich AI workflows rarely fail because of a single slow component. They become painful when orchestration, retrieval, model calls, and post-processing are chained without latency budgets or ownership. That creates a user experience problem, but it also becomes a resilience problem when agents sit inside operational workflows, handle sensitive data, or make repeated decisions that affect access, ticketing, or customer actions. The NIST Cybersecurity Framework 2.0 is useful here because it pushes teams to treat performance, reliability, and governance as part of security outcomes, not separate concerns.

For security leaders, the real issue is not that one model call is expensive. It is that multi-step systems amplify the latency of every dependency: policy checks, tool authentication, vector searches, guardrails, and output validation. If any one of those steps is tuned for accuracy only, the cumulative delay can make the system unusable and encourage unsafe shortcuts. In agentic AI, slow systems also create pressure to loosen controls, reuse stale context, or skip verification, which is a security regression disguised as optimisation. In practice, many security teams encounter latency complaints only after users have already created shadow workflows or bypassed the intended agent path.

How It Works in Practice

Multi-tool AI systems slow down because each step adds sequential overhead. A single user prompt may trigger intent classification, policy evaluation, retrieval, model inference, tool execution, and response synthesis. If those steps happen one after another, the total delay is the sum of all the small delays, plus retries and queueing. That is why systems often feel fast in demos and slow in real operations, especially when live APIs, identity checks, or approval gates are introduced.

Operationally, the main sources of delay are predictable:

• Network round trips between the orchestrator, model, and external tools.
• Context growth, where large prompts increase token processing time and cost.
• Repeated model calls for planning, validation, and summarisation.
• Tool latency, especially when the agent waits on slow SaaS APIs or internal services.
• Guardrails and post-processing that validate outputs before release.

Good practice is to instrument each stage separately, not just measure end-to-end request time. Teams should record where time is spent, which tools are slowest, and whether the agent is making unnecessary sequential calls. This is especially important where the system touches identities, secrets, or privileged actions, because authentication and approval steps can become hidden bottlenecks. For AI systems that integrate retrieval or memory, the quality of the index and the scope of the context window also matter because poor retrieval can force extra model calls. Guidance from NIST AI Risk Management Framework supports treating these tradeoffs as part of system design and monitoring, not as a late-stage tuning exercise.

Teams can usually improve performance by collapsing redundant steps, caching stable results, parallelising independent calls, and limiting the agent’s freedom to re-plan on every turn. Where tool access is governed through NHI-style controls, pre-authorising safe actions and reducing repeated credential exchanges can help. These controls tend to break down when the workflow depends on many external SaaS APIs with inconsistent response times because the agent cannot control downstream queueing or timeout behaviour.

Common Variations and Edge Cases

Tighter orchestration often improves reliability but increases design and governance overhead, so organisations need to balance speed against control depth. In some environments, that tradeoff is worth it. In others, especially customer-facing chat or analyst support, aggressive chaining creates a latency tax that users immediately notice.

There is no universal standard for how much latency is acceptable in agentic systems, but current guidance suggests setting stage-level budgets and using them to decide when to stop planning and return a partial answer. For retrieval-heavy systems, the bottleneck may be the search layer rather than the model. For action-heavy systems, the bottleneck is often approval, identity proofing, or tool authentication. Where autonomous tools can trigger sensitive actions, security teams should also evaluate whether the system is calling tools too often because the policy layer is too vague.

Edge cases appear when multiple models are involved, such as one model drafting, another checking policy, and a third rewriting output. That can improve quality, but it frequently multiplies cost and delay without improving outcomes. The best implementations keep the critical path short and reserve extra checks for high-risk actions. For agentic systems, the practical rule is simple: if a step does not change the decision or reduce risk, it should be removed, cached, or moved off the critical path.

Related resources from NHI Mgmt Group

• When do AI agents become an NHI governance problem instead of an automation tool?
• How should organisations offboard a shadow AI tool that was connected to company systems?
• How should organisations govern AI systems under multiple regulatory regimes?
• How should security teams govern AI gateway traffic that carries prompts and tool calls?