Threat modelling is the cheapest point at which to make an AI agent safe, yet most agents reach production with credentials and tools that nobody has examined as a whole. Before an agent receives credentials, a two-hour workshop can show what it will read, what it can do, who it acts for and how an attacker could turn one into the other. This guide gives security, IAM and engineering teams a repeatable method for threat modelling AI agents. It combines the four questions of classic threat modelling, the layered view of the CSA MAESTRO framework, the OWASP Top 10 for Agentic Applications and the MITRE ATLAS technique catalogue, with a strong focus on identity, with a worked example, metrics and a roadmap.
Key takeaways
- Model before credentials are issued. Threat model every agent before it gets production access, and again when its tools, data, model, autonomy or delegation model change.
- Four questions frame the work. Ask what the agent can read, what it can do, who it acts for and what can reach it; they are the agent version of "what are we working on?" and "what can go wrong?".
- The identity map sets the blast radius. Every credential, token and delegated scope in the design decides how bad each threat can get, so map them all, with scope, lifetime and storage.
- Use existing catalogues, not new categories. MAESTRO layers make sure nothing is missed, OWASP ASI01 to ASI10 give the threat checklist, STRIDE covers each data flow and ATLAS names attacker techniques.
- End with testable controls outside the model. Each finding needs a control enforced by identity, policy or architecture, an owner and a test, then red teaming before go-live.
Frameworks you can draw on
The Threat Modeling Manifesto reduces the practice to four questions: "What are we working on? What can go wrong? What are we going to do about it? Did we do a good enough job?" OWASP's threat modelling guidance adds that a threat model should record assumptions that "can be checked or challenged in the future" and a way of validating the model, which matters for agents because their tools and models change often.
| Framework | What it gives you | Use it for | Limits for agents |
|---|---|---|---|
| CSA MAESTRO (Multi-Agent Environment, Security, Threat, Risk and Outcome) | A seven-layer reference architecture with a threat landscape per layer, plus cross-layer threats | Decomposing the system so no layer is skipped | Threat lists are broad; you still need to tailor them to the design |
| OWASP Top 10 for Agentic Applications 2026 | Ten agent risks, ASI01 Agent Goal Hijack to ASI10 Rogue Agents, with examples and mitigations | The threat checklist at each layer | A prioritised list, not a method |
| STRIDE | Six threat types: spoofing, tampering, repudiation, information disclosure, denial of service, elevation of privilege | Each component, data flow and trust boundary | CSA notes it lacks AI-specific threats and multi-agent interaction |
| MITRE ATLAS | Adversary tactics and techniques against AI systems (AML.T IDs) | Naming attacker techniques, red team plans, detection content | Technique-level; does not prioritise for your design |
| OWASP Non-Human Identities Top 10 (2025) | Ten risks in the credentials systems use, NHI1 to NHI10 | Checking each credential on the identity map | Not agent-specific; pair with ASI03 |
CSA's MAESTRO paper reviews STRIDE, PASTA, LINDDUN, OCTAVE, Trike and VAST and concludes that each needs augmenting for agentic AI: it calls STRIDE "a good starting point" that "doesn't address the unique challenges of AI agents". The layers, as described in the paper, run from foundation models (layer 1), data operations (layer 2) and agent frameworks (layer 3) through deployment and infrastructure (layer 4), evaluation and observability, and a security and compliance layer that "cuts across all other layers", to the agent ecosystem where agents meet users and other agents. For model-level attacks such as poisoning and evasion, NIST AI 100-2 E2025 gives a taxonomy of adversarial machine learning attacks and mitigations. You do not need every framework. A workable combination is MAESTRO layers for coverage, OWASP ASI as the checklist, STRIDE per data flow and an identity map to size impact. For the general method, see the Threat Modelling Guide; for the attacker catalogue, see the MITRE ATLAS Guide.
Before the workshop
- Participants: the agent's owner, a developer who knows the design, an IAM or identity engineer, a security architect, and someone who knows the business process and the data. Nobody on a typical agent team sees both the prompts and the entitlements.
- Inputs: the agent's purpose; an architecture diagram; the list of tools, MCP servers and APIs; data sources and memory stores; the identity and credential design; the autonomy level and approval points; and the model and framework versions.
- Pre-work: export the agent's actual entitlements from the identity provider, cloud IAM and each SaaS platform, not the intended ones. The gap is often the first finding.
- Time: 90 minutes to two hours for a single agent; a half day for multi-agent systems or agents that cross organisational boundaries.
The method, step by step
Step 1: Define scope and purpose
Write one sentence on what the agent is for and one on what it must never do. The second sentence is often the most useful output of the whole workshop, because every "must never" becomes a control that has to be enforced outside the model. Record the autonomy level too. OWASP's agentic Top 10 recommends "Least-Agency", advising organisations "to avoid unnecessary autonomy", so ask whether the agent needs each capability at all.
Step 2: Draw the system and its trust boundaries
Draw the agent, its model provider, orchestrator, memory, tools and MCP servers, the systems behind them, the users who interact with it, and any other agents. OWASP's cheat sheet says a model should give "a clear view of trust boundaries, data flows, data stores, processes, and the external entities". Mark every point where untrusted content enters, such as email, web pages, documents, tickets, tool output and messages from other agents. For agents this is the most important line on the diagram: ASI01 exists because agents and their models "cannot reliably distinguish instructions from related content".
Step 3: Map identities and credentials
For every arrow in the diagram, record who authenticates to whom and how. Use a table like this one and fill it from real configuration, not memory.
| Credential | Held by | Grants | Lifetime and storage | Questions to ask |
|---|---|---|---|---|
| Agent's own identity | Agent runtime | Its own permissions in each platform | Workload identity, certificate or secret | Is it unique to this agent, or reused (NHI9)? |
| Delegated user token | Agent, on a user's behalf | The user's scopes, or a subset | Minutes to days; often cached | Is it narrowed to the task, and does it expire with the task? |
| Tool or MCP server credential | The tool, not the agent | Backend API access | Often a static key in config | Could the server use it for anything else? |
| Model provider key | Agent or gateway | Inference and spend | Static API key | Is it behind a gateway with budgets? |
| Reachable credentials | Files, environment variables, cloud metadata | Whatever they were created for | Long-lived (NHI7) | Could the agent find and use them? |
The last row is the one teams forget. ATLAS catalogues Credentials from AI Agent Configuration (AML.T0083) and AI Agent Tool Credential Harvesting (AML.T0098), and the PocketOS Database Deletion 2026 shows the non-malicious version: an agent working in staging found an account-scoped token in an unrelated file and used it to delete production data. This map is what turns a list of threats into a list of consequences. See the Secrets Management Guide for removing reachable secrets.
Step 4: Identify threats layer by layer
Walk through the MAESTRO layers and, at each, ask which OWASP ASI risks and STRIDE categories apply. MAESTRO also asks you to look for cross-layer threats, such as an infrastructure compromise in layer 4 used to poison the data store in layer 2. The prompts in the next section help keep the conversation concrete.
Step 5: Rate the risks
For each threat, rate likelihood (how exposed is the entry point, how easy is the attack) and impact (what the credentials involved allow, how reversible the result is, how sensitive the data). Keep ratings simple (high, medium, low) and agree them in the room. Consistency between workshops matters more than precision. Treat any threat that combines an untrusted input channel with a write or delete capability as high until proven otherwise.
Step 6: Choose controls
For each high and medium risk, choose controls, preferring those that work whatever the model decides:
- Reduce access: narrower scopes, task-scoped and time-bound tokens, separate identities per agent, removing tools that are not needed. ASI03's first mitigation is to "issue short-lived, narrowly scoped tokens per task".
- Add enforcement: policy checks at a tool gateway, per-action authorisation for privileged steps, and human approval for irreversible actions.
- Isolate: separate agents, memory or credentials for reading untrusted input and for privileged action; wipe state between tasks and users.
- Detect and respond: specific log fields, alerts on unusual tool calls and scope requests, and a tested way to revoke the agent's credentials quickly.
Each control needs an owner and a way to test it. See the AI Agent Authorisation Guide and the OAuth Token Exchange and Delegation Guide for scoping delegated access.
Step 7: Validate and maintain
- Red team the agent against the top threats before go-live: indirect prompt injection through each input channel, attempts to escalate privilege, attempts to find and use credentials, and attempts to exfiltrate data through write-capable tools.
- Store the threat model with the agent's registry entry, where reviewers and incident responders can find it.
- Re-run it when tools, data sources, model, autonomy level or delegation model change. OWASP lists new features, security incidents and architectural changes as triggers for updating any threat model.
See Red Teaming AI Agents for Identity Abuse for test design.
Threat prompts by layer
ATLAS IDs were checked against the 2026.09 ATLAS data release; use them to name test cases and detections, not as proof of coverage.
| Area (MAESTRO layer) | Prompt for the room | OWASP ASI | ATLAS technique |
|---|---|---|---|
| Inputs (agent ecosystem) | Where could an attacker place instructions the agent will read? | ASI01 | AML.T0051.001 LLM Prompt Injection: Indirect |
| Foundation models (layer 1) | What happens if the model follows those instructions, or simply makes a mistake? | ASI01, ASI09 | AML.T0051 LLM Prompt Injection |
| Data operations and memory (layer 2) | Can one user's input influence another user's session or future tasks? | ASI06 | AML.T0080.000 AI Agent Context Poisoning: Memory; AML.T0070 RAG Poisoning |
| Agent frameworks and tools (layer 3) | Which tools could cause harm if misused? Can untrusted input reach a shell, query or file path? | ASI02, ASI05 | AML.T0053 AI Agent Tool Invocation; AML.T0101 Data Destruction via AI Agent Tool Invocation |
| Supply chain (layers 3 and 4) | Which tools, MCP servers, packages and models come from third parties, and how are they verified? | ASI04 | AML.T0110 AI Agent Tool Poisoning |
| Identity (cross-layer) | What is the worst action possible with each credential on the map? Could the agent find credentials it was not given? | ASI03 | AML.T0083 Credentials from AI Agent Configuration; AML.T0098 AI Agent Tool Credential Harvesting |
| Other agents (agent ecosystem) | How are other agents authenticated, and what happens if one lies or fails? | ASI07, ASI08, ASI10 | AML.T0081 Modify AI Agent Configuration |
| Outputs (frameworks and ecosystem) | Which write-capable tools could carry data out of the organisation? | ASI02 | AML.T0086 Exfiltration via AI Agent Tool Invocation |
| Evaluation and observability | Would you know it happened, and could you prove what the agent did and for whom? | ASI10 | AML.T0084.001 Discover AI Agent Configuration: Tool Definitions |
For the deeper controls behind each row, see the Prompt Injection Guide, the AI Agent Memory Security Guide, the AI Agent Tool and Plugin Security Guide and the MCP Security Guide.
Worked example: an IT service desk agent
Purpose: reads incoming support emails and tickets, answers common questions, and can unlock accounts and reset multi-factor authentication for employees.
Must never: change permissions, reset credentials for privileged accounts, or act on requests it cannot verify came from the account owner.
Identities on the map: the agent's service identity; a mailbox read and send permission; a ticketing MCP server token; an identity provider admin API credential scoped to account unlock and MFA reset; a model provider key.
| Threat | Layer / OWASP | Impact via identity | Rating | Controls |
|---|---|---|---|---|
| Attacker emails the desk pretending to be an executive and asks for an MFA reset | Inputs, model / ASI01, ASI09 | Agent's IdP credential can reset MFA, enabling account takeover | High | Verify requester out of band before any reset; block resets for privileged and executive accounts; human approval for all MFA resets |
| Hidden instructions in an email tell the agent to forward other tickets externally | Inputs / ASI01, ASI02 | Mailbox send permission allows exfiltration | High | Restrict send to internal domains; separate reading untrusted email from sending; alert on external sends |
| IdP admin credential broader than needed | Identity / ASI03 | Could change groups or roles if misused | High | Custom IdP role limited to unlock and MFA reset for standard users; short-lived token via workload identity |
| Malicious update to community ticketing MCP server | Ecosystem / ASI04 | Server holds ticketing token and sees all tickets | Medium | Use approved, pinned server; gateway with change detection; scoped token |
| One user's conversation influences answers to another | Memory / ASI06 | Disclosure of another user's ticket data | Medium | Per-user memory isolation; no raw input stored in long-term memory |
| Actions not attributable during an investigation | Observability / repudiation | Cannot show who requested a reset | Medium | Log requester, verification result, approver, action and credential ID |
The pattern is typical: the most serious threats combine an untrusted input channel with a powerful credential. Reducing the credential's scope and adding a verification step removes most of the risk, whatever the model does. The first two rows are not hypothetical. In EchoLeak 2025, researchers showed that a crafted email retrieved as context could steer Microsoft 365 Copilot into exposing data it reached with the user's access (Microsoft reported no exploitation in the wild), and the postmark-mcp Malicious MCP Server 2025 quietly blind-copied every email an agent sent, which is the fourth row. See the Account Recovery and Help Desk Security Guide.
Common mistakes
- Modelling the model rather than the system: spending the session on jailbreaks and ignoring what the agent's credentials allow.
- Leaving identities off the diagram: impact is guessed rather than known, and reachable credentials in files or environment variables are missed, as in PocketOS.
- Controls that live only in the system prompt: instructions are not permissions. In the Replit AI Agent Database Deletion 2025, an agent deleted a live database during a declared code freeze, against explicit instructions.
- Trusting tools because they are popular: a third-party MCP server or package runs with the agent's delegated access; one changed line was enough in postmark-mcp.
- Treating the model as a one-off document: agents gain tools and scopes quickly, and a model that is never revisited describes a system that no longer exists.
- Skipping multi-agent interactions: each agent was modelled separately, so confused-deputy paths between them, which ASI03 describes as "Cross-Agent Trust Exploitation", were never examined. See the Multi-Agent and A2A Security Guide.
Framework mapping
This is an indicative alignment prepared by NHI Mgmt Group, not an official mapping. Control IDs were checked against NIST CSWP 29 (CSF 2.0), the NIST SP 800-53 Rev. 5 catalogue, the CIS Controls (control level) and a published ISO/IEC 27001:2022 Annex A list.
| Workshop activity | NIST CSF 2.0 | NIST SP 800-53 Rev. 5 | ISO/IEC 27001:2022 Annex A | CIS Controls v8.1 | OWASP |
|---|---|---|---|---|---|
| Model the system, data flows and trust boundaries | ID.AM-02, ID.AM-03 | PL-8, SA-17, CM-8 | A.5.9, A.8.27 | 1, 2 | ASI01 (input channels) |
| Identify and rate threats | ID.RA-03, ID.RA-04, ID.RA-05 | RA-3, SA-11(2) | A.5.7, A.8.25 | 16 | ASI01 to ASI10 |
| Map identities, credentials and delegated scopes | PR.AA-01, PR.AA-05 | AC-2, AC-6, IA-5, IA-9 | A.5.16, A.5.17, A.5.18, A.8.2 | 5, 6 | ASI03; NHI5, NHI7, NHI9 |
| Assess third-party tools, MCP servers and models | GV.SC-07, ID.RA-09 | SR-3, SR-6, SA-9 | A.5.19, A.5.21 | 15, 16 | ASI04; NHI3 |
| Enforce controls outside the model | PR.AA-05, PR.PS-01 | AC-3, AC-5, CM-7, SI-10 | A.5.15, A.8.3 | 4, 6 | ASI02, ASI05, ASI09 |
| Log and attribute agent actions | PR.PS-04, DE.CM-09 | AU-2, AU-3, AU-10, SI-4 | A.8.15, A.8.16 | 8 | ASI10 |
| Validate by testing and re-model on change | ID.IM-02, ID.RA-07 | CA-8, SA-11 | A.8.29, A.8.32 | 16, 18 | ASI08 |
The CIS Controls Assessment Specification lists "Conduct Threat Modeling" as safeguard 16.14 under Application Software Security, which is the closest single anchor for this practice. See the Control Mapping Guide.
Metrics that show progress
| Metric | What it shows | Target direction |
|---|---|---|
| Production agents with a current threat model | Coverage of the practice across the agent estate | Up, towards all agents |
| Agents that received credentials before a threat model | Whether the review gate is real | Down, towards zero |
| High-rated findings with a tested control | Whether findings turn into enforcement | Up |
| Credentials on identity maps that are long-lived or shared | Residual blast radius (NHI7, NHI9) | Down |
| Red team tests passed per top threat | Whether controls hold under attack | Up |
| Incidents involving a threat that was not in the model | Blind spots in the method | Down; feed each into the prompt list |
See the AI Agent Observability and Incident Response Guide for the logs that make the last two metrics measurable.
Implementation roadmap
| Phase | Actions | Done when |
|---|---|---|
| First 30 days | Adopt the template (purpose, must-never, diagram, identity map, threat table); run workshops for the three agents with the most powerful credentials; record findings with owners | Three models stored with registry entries; high findings have owners and dates |
| Days 31 to 90 | Make a threat model a condition of issuing production credentials; add ATLAS-named red team cases for the top threats; train two facilitators per business unit | No new agent gets production credentials without a model; first red team results recorded |
| Days 91 to 180 | Automate re-modelling triggers from changes to tools, scopes and models; review identity maps against live entitlements quarterly; report the metrics above | Stale models flagged automatically; metrics reported to the AI governance forum |
See the AI Agent Deployment Checklist for the go-live gate and the Identity Governance for AI Agents Guide for ownership and reviews.
Practitioner checklist
- Threat model each agent before issuing production credentials.
- Record purpose, autonomy level and "must never" statements.
- Draw trust boundaries and every untrusted input channel.
- Map every identity and credential, with scope, lifetime and storage, from live configuration.
- Look for credentials the agent could reach but was not given.
- Walk MAESTRO layers with OWASP ASI01 to ASI10 as the checklist and STRIDE per data flow.
- Rate risks by likelihood and by impact through identity, including reversibility.
- Choose controls outside the model, each with an owner and a test.
- Name red team cases with ATLAS technique IDs and run them before go-live.
- Store the model with the agent's registry entry and re-model on significant change.
Standards and references
- Cloud Security Alliance: Agentic AI Threat Modeling Framework: MAESTRO
- Cloud Security Alliance: MAESTRO Threat Analyzer repository
- OWASP GenAI Security Project: OWASP Top 10 for Agentic Applications for 2026
- OWASP: Top 10 Non-Human Identities Risks 2025
- MITRE ATLAS: ATLAS data repository
- NIST: AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
- Threat Modeling Manifesto
- OWASP: Threat Modeling
- OWASP Cheat Sheet Series: Threat Modeling Cheat Sheet
- Microsoft Learn: Microsoft Threat Modeling Tool threats (STRIDE)
- NIST: The NIST Cybersecurity Framework (CSF) 2.0
Related NHI Mgmt Group resources: Agentic AI Security Guide · AI Agent Authorisation Guide · Prompt Injection Guide · Multi-Agent and A2A Security Guide · Top 10 Agentic AI Identity Issues