How should security teams triage AI agent execution graphs in production?

Why This Matters for Security Teams

AI agent execution graphs are not just observability artefacts. They are evidence of what the agent actually attempted, which tools it touched, which decisions it escalated, and where human or policy boundaries were crossed. Security teams that treat graphs as product telemetry alone miss the operational question: did the agent behave within its intended authority, or did it drift into unsafe autonomy?

That distinction matters because agentic systems fail in ways that look normal at first. A graph can show a routine chain of tool calls while hiding prompt injection, poisoned retrieval, or a gradual expansion in scope across retries. The NIST AI Risk Management Framework is useful here because it pushes teams to evaluate not only outputs but the conditions under which the system produced them, including traceability, accountability, and ongoing monitoring.

Current guidance suggests triage should focus on whether the graph stayed inside approved workflows, whether it probed protected resources, and whether it completed an action that changed state, exposed data, or altered permissions. In practice, many security teams encounter the real risk only after an agent has already chained together several seemingly benign steps into an unintended action.

How It Works in Practice

Effective triage starts by comparing each execution graph to a baseline of expected paths for that agent, that task type, and that environment. The baseline should capture normal tool sequence, expected latency, common retries, approved destinations, and known fallback behaviour. A graph that stays inside the baseline but shows noise, retries, or low-risk deviations can often be documented. A graph that attempts new tools, unusual prompts, or repeated boundary tests should move to investigation. A graph that reads sensitive data, writes to external systems, or invokes privileged actions should be treated as a security event.

Teams usually get the most value when they classify graph elements by risk, not by visual complexity. Useful triage dimensions include:

• Identity and authority: which agent identity, secret, or delegated token was used
• Tool scope: whether the agent touched approved tools only or expanded into adjacent systems
• Data sensitivity: whether the graph accessed customer data, secrets, or regulated content
• Control crossing: whether the agent moved from analysis to execution, or from read-only to write action
• Attack pattern fit: whether the sequence resembles prompt injection, tool misuse, or exfiltration

The MITRE ATLAS adversarial AI threat matrix helps security teams map graph behaviour to known adversarial techniques, while the OWASP Agentic AI Top 10 is useful for thinking about tool misuse, excessive agency, and unsafe delegation. If the graph indicates the agent reached a state-changing boundary, such as ticket creation, fund movement, code deployment, or policy override, triage should shift from analysis to incident handling. These controls tend to break down when graphs are incomplete, tool logging is inconsistent, or multiple agents share the same runtime identity because attribution becomes ambiguous.

Common Variations and Edge Cases

Tighter graph triage often increases analyst workload and storage overhead, requiring organisations to balance detection fidelity against operational cost.

Not every unusual graph is malicious, and not every malicious graph looks unusual. Current guidance suggests that teams label this distinction clearly in playbooks rather than relying on intuition. Long-running agent jobs, multi-step research workflows, and recovery logic after tool failures can all create graphs that look suspicious but are operationally normal. The reverse is also true: a short, clean-looking graph may still conceal a high-risk action if the agent used a privileged token or called a sensitive API once.

Edge cases often appear when agents are embedded in business processes with weak control boundaries. For example, an agent that can draft content, submit requests, and approve its own outputs may show a perfectly linear graph while still violating separation of duties. In those cases, the graph should be evaluated alongside the authority model, not in isolation. The NIST AI Risk Management Framework and the CSA MAESTRO agentic AI threat modeling framework both reinforce that governance, provenance, and control design must be reviewed together. There is no universal standard for graph severity scoring yet, so mature teams document local thresholds and revisit them after incidents, red-team exercises, or major workflow changes.

Related resources from NHI Mgmt Group

• How should security teams evaluate AI agent trust before production use?
• How should security teams detect risky AI agent behaviour in production?
• How should security teams trace AI agent failures in production?
• How should security teams limit the risk from AI agents that have access to production systems?