Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

Agentic AI risk progression: what changes when models can act?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 9236
Topic starter  

TL;DR: The OWASP Top 10 for Agentic Applications reframes agentic AI security as a progressive breach model in which prompt injection, memory poisoning, tool misuse, and identity abuse can compound into system-wide failure once models are allowed to act, according to Lakera. The security shift is from filtering outputs to containing amplification and blast radius.

NHIMG editorial — based on content published by Lakera: The Progressive Breach Model Behind the OWASP Top 10 for Agentic Applications

By the numbers:

Questions worth separating out

Q: How should security teams govern AI agents that can call tools and use credentials?

A: Security teams should govern AI agents as delegated actors, not just chat interfaces.

Q: Why do AI agents create a different risk model from traditional LLM applications?

A: AI agents create a different risk model because they can turn manipulated context into action.

Q: What breaks when agents can retain memory and reuse retrieved context?

A: What breaks is the assumption that the attack ends when the session ends.

Practitioner guidance

  • Map agent-to-tool trust boundaries Inventory every tool, workflow, and data source an agent can reach, then classify which of those paths can change business state without human review.
  • Separate intent from execution rights Give agents the minimum operational permissions needed for a task and keep high-risk actions behind a distinct approval or orchestration layer.
  • Instrument memory and retrieval inputs Log which documents, messages, and retrieved items can influence planning so you can trace when a poisoned context entry becomes actionable.

What's in the full article

Lakera’s full article covers the technical detail this post intentionally leaves for the source:

  • The article breaks down the OWASP Agentic Top 10 category by category, including the differences between prompt injection, goal hijack, memory poisoning, tool misuse, and cascading failures.
  • It expands on concrete agentic attack paths such as poisoned documents, malicious tool outputs, and inter-agent propagation patterns that this post only frames at a high level.
  • It includes the vendor’s own examples and internal research references on how agentic threats manifest in real environments.
  • It shows how Lakera maps the progression from compromise to loss of containment across the agent lifecycle.

👉 Read Lakera’s analysis of the OWASP Top 10 for agentic applications →

Agentic AI risk progression: what changes when models can act?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 8675
 

Autonomy changes the breach model, not just the attack surface. Traditional LLM security assumes the main failure is unsafe output. That assumption fails when the system can plan, call tools, and act under real credentials. The implication is that identity governance has to be judged by how much delegated power a compromised objective can convert into action, not by prompt filtering alone.

A few things that frame the scale:

  • 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
  • 52% of companies can track and audit the data their AI agents access, which means 48% cannot fully support compliance or breach investigation, according to the same SailPoint research.

A question worth separating out:

Q: How do organisations contain cascading failures in agentic systems?

A: Organisations contain cascading failures by treating agent interactions as a trust boundary. Shared memory, internal messages, and dynamic tool discovery should be verified, scoped, and isolated so one compromised agent cannot influence many others. The goal is to limit propagation before a bad decision becomes a system-wide failure.

👉 Read our full editorial: OWASP agentic applications top 10 as a progressive breach model



   
ReplyQuote
Share: