TL;DR: AI agent slowdowns can stem from reasoning depth, backtracks, or failed tools, according to Codenotary. Monitoring has to tie task execution, tool behaviour, and spend together if teams want to explain what changed.
Editorial analysis by NHI Mgmt Group, based on content published by Codenotary: “AI Agent Performance Monitoring with OpenTelemetry”.
By the numbers:
- The post cites 11.5 seconds at P99 as an example of agent latency that still leaves the root cause hidden.
- The post says a tool in the example failed 32% of the time, which changed the agent’s overall performance profile.
- Codenotary says the fleet-wide tool-call error rate was 32.98% across 96,759 tool calls in the window.
Key questions
Q: Why do long chain-of-thought outputs create operational risk in production AI systems?
A: Long chain-of-thought outputs increase token usage, latency, and GPU cost, which can make user-facing systems slower and more expensive.
Q: What are the signs that an AI agent is stuck rather than just busy?
A: Repeated backtracks, many failed subgoals, and a high count of tool calls with poor success rates are the main signs.
Q: How should security teams monitor AI agents without relying on traditional APM alone?
A: Security and platform teams should monitor AI agents with spans that expose reasoning steps, tool calls, errors, and cost, not just request latency.
Practitioner guidance
- Instrument thought, tool, and cost spans Emit spans for reasoning steps, tool calls, durations, status, and token cost so the agent’s internal loop is visible in one trace.
- Track reasoning depth and backtracks Use reasoning depth and backtrack count as primary indicators of agent thrash, especially when latency alone does not explain poor outcomes.
- Review high-error tools as governance signals Treat tools with elevated error rates as control problems, because the agent may be repeatedly relying on a failing dependency or a broken workflow.
Bottom line: Traditional APM does not explain AI agent slowdowns well because it cannot see reasoning depth, backtracks, or tool-level failure patterns.
Glossary terms: AI-Agent Monitoring, Reasoning Depth, Backtrack, Span-Level Costing
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
Agent observability is becoming a governance control, not a dashboard feature. Once an agent can reason, retry, and chain tools, the question is no longer whether the service responded. The question is whether its internal decision path is auditable enough to explain cost, latency, and failure. That shifts observability into identity and authority territory because every tool call reflects some delegated action. Practitioners should treat trace fidelity as part of control design, not just engineering convenience.
A question worth separating out:
Q: How should organisations govern external tools used by AI agents?
A: Organisations should review external tools as security inputs, not convenience features. Each tool needs ownership, approval, metadata inspection, and ongoing monitoring for hidden instructions or unexpected behaviour. If an AI agent can act on a tool, then the tool’s provenance and control status should be governed like any other sensitive integration.
👉 Read our full editorial: AI agent performance monitoring needs reasoning and cost signals