AI threat modeling for agentic systems needs runtime controls

At a glance

What this is: The article argues that agentic AI needs a different threat modelling approach because runtime decisions, tool use, and connected systems expand the attack surface beyond conventional application assumptions.

Why it matters: This matters because IAM, NHI, and agent governance teams need to model what agents can access, who can instruct them, and how their actions are verified before those permissions create exposure.

👉 Read Lasso Security's guide to building an AI threat modelling process for agentic systems

Context

AI threat modelling for agentic systems starts with a simple problem: the model is no longer just producing text, it is making decisions that affect tools, data, and downstream business actions. Traditional application threat models assume relatively stable paths and predictable inputs, but agentic workflows can change during execution, which makes identity, permissions, and trust boundaries far more important than static diagrams suggest.

For IAM and NHI practitioners, the important point is that an AI agent can function as a new access path. Once an agent can query records, update systems, send messages, or trigger workflows, its credentials, scopes, and delegation paths need the same scrutiny applied to privileged service accounts and other non-human identities. That intersection is where threat modelling becomes governance, not just design review.

Key questions

Q: How should security teams apply threat modeling to AI systems?

A: Security teams should model AI systems as a combination of data pipelines, inference surfaces, outputs, and connected workflows. That means mapping what the model can read, what it can generate, and what downstream actions those outputs can trigger, then adding controls for access, validation, monitoring, and approval where risk is highest.

Q: Why do AI agents create more governance risk than ordinary integrations?

A: AI agents can connect quickly, run continuously, and accumulate broad permissions across multiple services. That combination makes ownership blur and scope drift more likely, so the real risk is not the tool itself but the uncontrolled access path it creates across enterprise systems.

Q: What breaks when AI agents are given broad inherited permissions?

A: Broad inherited permissions break the assumption that access is tied to a narrow business need. The result is larger blast radius, weaker accountability, and faster propagation of mistakes or abuse across connected systems. A single compromised or misconfigured agent can then touch far more data and workflows than the original task required.

Q: What frameworks should organisations use to assess agentic AI risk?

A: Use OWASP Agentic AI Top 10 for threat modelling, OWASP NHI guidance for credential and privilege governance, and Zero Trust principles for continuous verification of tools and identities. Together, they help teams evaluate agent behaviour, access boundaries, and trust at runtime instead of only at onboarding.

Technical breakdown

Why agentic AI breaks static threat modelling assumptions

Traditional threat modelling works best when the system path is known in advance. Agentic AI is different because the model can decide which tool to call, which retrieved context to trust, and what action to take next. That means the attack surface is not fixed at design time. It expands through prompts, retrieval, tool outputs, memory, and orchestration layers. A poisoned document or manipulated context can redirect execution without touching the application perimeter. The result is a dynamic risk profile that conventional diagrams understate.

Practical implication: Model the workflow as a runtime decision chain, not a static app, and review every trust boundary that can change execution.

How identity and permissions shape AI agent risk

Once an agent can act on CRM data, financial records, tickets, code, or email, it becomes part of the enterprise access layer. At that point, the core questions are identity questions: what can the agent access, what can it do, who can instruct it, and how are its actions verified. If those controls are unclear, the agent can inherit excessive agency, bridge systems that were never meant to be connected, and widen blast radius when manipulated. This is where agent governance overlaps with NHI governance in a practical way.

Practical implication: Scope every agent to task-specific permissions and treat its credentials, delegation, and approvals as governed access, not generic automation.

Why runtime behaviour and cross-system dependencies matter

Agentic workflows are not just about the model prompt. They also depend on tool chaining, approval logic, fallback behaviour, and the way one output becomes another tool's input. That creates cross-system dependencies that attackers can exploit through indirect prompt injection, unsafe tool use, or poisoned retrieval data. Static threat models often miss these chains because they focus on single controls rather than the sequence of decisions and handoffs. Runtime behaviour baselines and deviation detection help expose when an agent is acting outside its intended pattern.

Practical implication: Map tool chains, approval paths, and abnormal execution patterns so deviations can be detected before they become business actions.

NHI Mgmt Group analysis

Agentic AI governance is becoming an identity problem as much as an AI problem. The article's core point is that once an agent can retrieve data, choose tools, and trigger actions, its permissions matter as much as its model quality. That moves the discussion from prompt safety to access governance, delegation, and verification. In practice, this means IAM and NHI controls now sit inside the AI risk model, not beside it.

Runtime behaviour is the named concept security teams need to track. Static architecture diagrams cannot capture tool chaining, delegated actions, or context-driven decision changes, so the real control gap is the mismatch between planned workflow and executed workflow. The article shows why threat models must be updated continuously as integrations, memory, and agent capabilities evolve. Practitioner conclusion: model behaviour drift as a first-class governance risk.

Excessive agency is the failure mode that most directly translates into enterprise exposure. When an AI agent can act beyond its intended purpose, the issue is not only malicious prompt content, but also over-broad permissions, weak instruction boundaries, and unclear approval rules. That creates a direct overlap with least-privilege thinking in identity security. Practitioner conclusion: treat agent authority as something that must be bounded, reviewed, and constrained.

Threat modelling for agentic AI has to connect technical risk to business consequence. The article correctly frames AI failures as privacy, finance, compliance, and operational events rather than isolated model defects. That broader lens aligns with NIST AI RMF and with governance expectations in regulated environments. Practitioner conclusion: if an AI workflow can affect records, money, or decisions, the threat model needs business owners, not only engineers.

MAESTRO and similar frameworks matter because they force layered analysis. The article's framework discussion shows that agent risk spans models, orchestration, tools, runtime, governance, and ecosystem dependencies. That layered view is where the field is moving, because single-control reviews miss multi-step manipulation paths. Practitioner conclusion: use layered frameworks to prevent blind spots between the model and the systems it touches.

What this signals

Runtime decisioning is the shift that will reshape AI governance programmes. Security teams should expect agent review to move closer to IAM-style control design, especially where agents can act on behalf of users or service processes. That makes delegation, approval, and access scoping central design decisions, not after-the-fact controls.

Agentic workflows will force closer alignment between AI governance and NHI governance. When an agent behaves like a privileged workload, the old split between model oversight and identity oversight stops holding. The practical programme response is to assess agent credentials, tool permissions, and approval chains together rather than in separate workstreams.

Threat modelling for AI will increasingly depend on standards-based evidence. Frameworks such as the NIST AI Risk Management Framework and MITRE ATLAS give practitioners a shared language for adversarial behaviour and governance. That matters because board, audit, legal, and security stakeholders need the same risk picture before deployment scales.

For practitioners

• Inventory every agent, tool, and MCP connection Build a complete register of GenAI applications, orchestration layers, memory stores, and connected business systems. Record what each agent can read, change, or trigger, then map those capabilities to business owners and approval paths.
• Map trust boundaries and runtime decision points Document where prompts, retrieved content, tool outputs, and downstream actions cross from untrusted to privileged contexts. Pay close attention to indirect prompt injection paths and to places where one tool's output becomes another tool's input.
• Constrain agent permissions to task scope Assign the minimum access needed for each workflow and separate read, write, and action privileges wherever possible. If an agent can send emails, update records, or query sensitive systems, those capabilities should be separately approved and reviewed.
• Baseline normal agent behaviour and alert on deviation Track expected tool-call sequences, approval steps, data access patterns, and fallback behaviour. Use those baselines to spot unusual execution paths, excessive agency, or suspicious context manipulation before the agent completes a harmful action.
• Update threat models as the agent lifecycle changes Refresh the model when prompts change, new integrations appear, memory is added, or a workflow expands into a new business process. Continuous monitoring should feed back into the threat model so controls stay aligned with current runtime behaviour.

Key takeaways

• Agentic AI changes threat modelling because runtime decisions, tool calls, and dynamic context expand the attack surface beyond conventional software assumptions.
• Identity, permissions, and verification become core controls once an AI agent can access business systems and perform actions on behalf of users or processes.
• The strongest programmes will treat AI threat modelling as a continuous governance process that updates as workflows, tools, and trust boundaries change.

Key terms

• Agentic AI Threat Modelling: Agentic AI threat modelling is the process of identifying how an AI system that can plan, call tools, and act on external systems could be manipulated or misused. It extends beyond prompts to include runtime behaviour, trust boundaries, identity, and downstream business impact.
• Excessive agency: A condition where an AI system is given more operational authority than its task requires. The risk is not just poor output. It is that mistakes, manipulation, or compromise can produce destructive actions at machine speed across the systems the agent can reach.
• Trust Boundary: A trust boundary is the point where one system’s authority should stop and another system’s authority should begin. For internal automation, weak trust boundaries let monitoring, remediation, and execution share privileges that should have remained separate.
• Runtime Behaviour Baseline: The expected pattern of activity for an identity while it is operating in production. It goes beyond entitlement lists by comparing actual actions, timing, and access paths, which is critical when valid credentials can still be abused.

What's in the full article

Lasso Security's full blog post covers the operational detail this post intentionally leaves for the source:

• Step-by-step guidance for inventorying agents, MCP connections, and orchestration layers across live environments.
• Framework-by-framework mapping that shows how to combine MAESTRO, MITRE ATLAS, and NIST AI RMF in practice.
• Control examples for runtime enforcement, deviation detection, and approval gating across agent workflows.
• Threat pattern breakdowns for prompt injection, tool poisoning, and poisoned retrieval that are not fully expanded here.

👉 Lasso Security's full post covers the layered framework mapping, runtime checks, and agent workflow controls in more detail.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, secrets management, and identity lifecycle controls for teams securing agentic and machine identities. It helps practitioners connect identity risk to the broader security programme they own.