Agentic AI threat modelling is the process of identifying how an AI system that can plan, call tools, and act on external systems could be manipulated or misused. It extends beyond prompts to include runtime behaviour, trust boundaries, identity, and downstream business impact.
Expanded Definition
agentic ai threat modelling is a structured way to analyse how an autonomous software entity with execution authority, tool access, and goal-seeking behaviour can be coerced, redirected, or abused. For NHI Management Group, the key distinction is that the threat surface is not limited to prompts or model outputs. It also includes tool invocation paths, memory and state, retrieval sources, identity and credential handling, permissions, and the business systems the agent can touch.
Usage in the industry is still evolving, and no single standard governs this yet. In practice, teams borrow concepts from traditional threat modelling, then adapt them for runtime orchestration, delegated authority, and indirect prompt injection. Frameworks such as the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 are useful references because they help teams reason about governance, failure modes, and abuse paths rather than treating the model as a standalone component.
The most common misapplication is reducing agentic AI threat modelling to prompt injection alone, which occurs when teams ignore the agent’s tool permissions, identity bindings, and downstream system access.
Examples and Use Cases
Implementing agentic AI threat modelling rigorously often introduces design friction, because each additional permission, integration, or memory feature increases capability as well as exposure. Security teams have to weigh automation gains against the cost of tighter control, more testing, and more explicit human approval paths.
- A customer-support agent can draft responses, but threat modelling reveals that a malicious document in its knowledge base could trigger unsafe tool calls or data exfiltration via retrieval-augmented generation.
- An internal operations agent can open tickets and update records, but analysis shows that overbroad service credentials let a compromised agent modify systems beyond its intended scope.
- A finance workflow agent can reconcile invoices, but threat modelling identifies a business logic path where manipulated input causes approval of fraudulent payments.
- A development agent can create pull requests, but the review process must account for poisoned dependencies, malicious instructions in code comments, and unauthorised repository actions, as reflected in guidance such as the CSA MAESTRO agentic AI threat modeling framework.
- A security analyst agent can correlate alerts, but the model may be steered into suppressing indicators if an attacker gains influence over its inputs or memory state, a concern also mapped in the MITRE ATLAS adversarial AI threat matrix.
Why It Matters for Security Teams
Agentic AI changes the security question from “what can the model say?” to “what can the system do when the model decides?” That shift matters because the real risk often sits in the orchestration layer: identity tokens, delegated access, API calls, approval bypasses, and chained actions across business platforms. For that reason, NHI Management Group treats agentic AI threat modelling as a control design activity as much as a risk assessment exercise.
It is especially relevant where agents use Anthropic’s first AI-orchestrated cyber espionage campaign report and CISA cyber threat advisories to understand real-world attacker tradecraft and emerging abuse patterns. In identity-heavy environments, the discipline also intersects with NHI governance, because agents frequently rely on machine identities, short-lived tokens, and scoped secrets that must be constrained, rotated, and monitored. If those bindings are weak, the agent can become a durable attacker foothold rather than a productivity layer.
Organisations typically encounter the full significance of agentic AI threat modelling only after an agent has already taken an unintended action, at which point the scope of delegated authority becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, CSA MAESTRO, MITRE ATLAS and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | AI RMF provides the core governance lens for identifying, measuring, and managing AI risks. | |
| OWASP Agentic AI Top 10 | OWASP Agentic AI Top 10 names common agent failure and abuse modes relevant to threat modelling. | |
| CSA MAESTRO | MAESTRO is a dedicated framework for agentic AI threat modelling and control design. | |
| MITRE ATLAS | ATLAS catalogs adversarial AI techniques that help model attack paths against AI systems. | |
| OWASP Non-Human Identity Top 10 | NHI guidance is relevant where agents depend on machine identities, secrets, and delegated access. |
Apply MAESTRO to enumerate trust boundaries, attacker paths, and required guardrails before deployment.
