Agentic AI security frameworks are being built from real incidents

At a glance

What this is: This is a Zenity commentary arguing that agentic AI security frameworks must be grounded in thousands of real incidents, not theoretical models, because recurring threat patterns are already visible in enterprise deployments.

Why it matters: It matters because IAM, NHI, and AI security teams need frameworks that map to actual agent behaviour, or they will keep missing the controls, detections, and governance steps that fail in production.

👉 Read Zenity's commentary on real incident data shaping agentic AI security frameworks

Context

Agentic AI security is moving faster than the governance models most enterprises rely on, which leaves teams trying to secure systems whose runtime behaviour is still poorly understood. In practice, the problem is not only AI risk, but the absence of a usable identity model for AI agents that can take actions, use tools, and trigger downstream effects across enterprise systems.

Zenity frames this as a research and community problem, but the underlying issue is broader. Security teams cannot govern autonomous behaviour with assumptions built for static applications or human-paced review cycles, and that creates a gap across AI security, NHI governance, and identity lifecycle controls.

Key questions

Q: How should security teams govern AI agents that can take actions on their own?

A: Treat the agent as an identity subject with runtime behaviour, not as a feature of the application. Define tool access, data scope, escalation paths, and logging requirements before deployment, then monitor whether the agent stays inside those boundaries during execution. Governance must connect IAM, PAM, and incident review because agent risk shows up in sequences, not just in authentication events.

Q: Why do agentic AI systems force security teams to rethink least privilege?

A: Because least privilege assumes the actor’s intent and access path are known at provisioning time. Autonomous agents can change path mid-session, combine tools dynamically, and create downstream effects that no static entitlement model fully predicts. That means privilege must be constrained by runtime context, not only by initial role assignment.

Q: What do security teams get wrong about agentic AI detection?

A: They often look for single bad events instead of chained behaviour. In agentic systems, risk appears when a sequence of tool calls, prompt changes, and data access decisions reveals goal hijacking or unintended escalation. Detection should therefore preserve context across the entire session and link actions to the same agent identity.

Q: Who is accountable when an AI agent causes unauthorized actions?

A: Accountability sits with the organisation that granted the agent access and defined its operating scope. If the governance model does not assign ownership for the agent’s data access, tool use, and escalation rights, the failure is organisational, not merely technical. That is why lifecycle ownership and review must be explicit from the start.

Technical breakdown

Why agentic AI security needs incident-led threat categories

Incident-led frameworks are built from repeated failure patterns rather than abstract speculation. In agentic systems, those patterns tend to cluster around goal hijacking, rogue actions, cascading failures, and supply chain exposure because the agent can combine prompts, tools, and data access at runtime. That combination creates security outcomes that look familiar to IAM teams, but behave differently once action timing and tool choice are not fully pre-scripted. The practical value of an incident-led taxonomy is that it maps to what defenders can actually observe and detect in production, not what they hope a lab model will capture.

Practical implication: Practitioners should anchor agentic AI controls in observed incident patterns, then map those patterns to detection and governance requirements before broad deployment.

How autonomous agent behaviour changes the identity problem

An AI agent is not just another workload account. When it can choose actions and tools at runtime, identity control becomes a problem of delegated decision-making, not just authentication or access assignment. That means the trust boundary moves from login events to session behaviour, from static permission sets to task-scoped execution, and from human review to machine-speed escalation paths. In identity terms, this is where traditional review and approval models start to lose coverage, because the risky behaviour happens inside the session rather than at provisioning time.

Practical implication: Security teams should treat agent runtime behaviour as an identity governance event, not only as an application security concern.

What detection engineering must account for in AI agent workflows

Detection for agentic systems has to understand sequences, not isolated alerts. A single action may look harmless, but a chain of tool calls can reveal goal hijacking, lateral movement through authorised services, or repeated execution driven by a poisoned instruction path. Zenity’s emphasis on low-noise, high-recall detection reflects a core operational truth: if alerting overwhelms analysts, the environment is not safer, it is simply less manageable. For AI security programmes, telemetry needs to preserve enough context to reconstruct intent, sequence, and downstream impact.

Practical implication: Build detections around multi-step agent behaviour and preserve context across tool calls, prompts, and access events.

Threat narrative

Attacker objective: The attacker seeks to hijack the agent’s goal and execution path so legitimate access produces unauthorized outcomes at machine speed.

1. Entry occurs when an AI agent is granted legitimate enterprise access and begins operating inside approved tools and data sources.
2. Escalation happens when the agent follows a poisoned objective, combines tools in unexpected ways, or expands its behaviour beyond the original operator intent.
3. Impact emerges when chained autonomous actions cause cascading failures, supply chain exposure, or unintended access to sensitive systems and data.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• Meta AI Instagram Account Takeover — 20,225 Instagram accounts hijacked via compromised Meta AI support chatbot with overprivileged access.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Incident-led agentic security is the only defensible way to build this category. Frameworks for agentic AI that are not grounded in repeated enterprise incidents will always lag the threat surface. Real operations reveal patterns such as goal hijacking, rogue actions, cascading failures, and supply chain abuse long before theory does. The practical conclusion is that AI security taxonomy must come from observed behaviour, not product architecture.

Access review processes assume privilege persists long enough to be reviewed, but autonomous agents can execute, adapt, and terminate within the same session. That assumption was designed for stable identities with observable review windows. It fails when the actor selects actions and tools at runtime without human approval gates. The implication is that governance must be redesigned around within-session behaviour, not periodic certification.

Agentic AI identity is already a governance problem, not just an application security problem. Once an agent can call tools, access data, and trigger downstream workflows, the control surface moves into IAM, PAM, and NHI territory. That makes the discipline broader than prompt filtering or model safety. Practitioners need to treat agent access as identity lifecycle, privilege, and accountability work.

Low-noise detection is now a governance requirement, not a tuning preference. Zenity’s emphasis on recall and signal quality reflects a deeper truth: if detections do not produce usable action, the programme cannot close the loop on real incidents. The field is heading toward evidence-based runtime governance, where telemetry quality determines whether the control exists in practice.

Runtime autonomy changes the meaning of least privilege. If the agent can change task path mid-execution, least privilege is no longer a static provisioning decision. It becomes a runtime constraint problem tied to intent, sequence, and tool choice, which is exactly where conventional identity models start to break down. Practitioners should expect their current entitlement model to understate the true blast radius.

From our research:

• 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, according to The State of Secrets in AppSec.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.
• That behaviour gap makes it harder to trust agentic workflows that depend on code-adjacent secrets, which is why practitioners should also read Ultimate Guide to NHIs , Lifecycle Processes for Managing NHIs.

What this signals

Runtime autonomy is turning identity governance into a continuous evidence problem. Access review cadences, approval gates, and static entitlement models were built for actors whose behaviour could be observed after the fact. When an AI agent can complete meaningful work inside one session, the control objective shifts from recertification to reconstructability, and that changes what logs, traces, and ownership records must exist.

The field is moving toward a model where agentic AI, NHI, and PAM controls converge around one question: can the organisation explain what the system did, why it had access, and who is accountable for it. That is where internal references like the 52 NHI Breaches Analysis remain useful, because they show how quickly access assumptions become breach assumptions.

identity blast radius: the practical reach of an agent’s access when one tool call can trigger several downstream systems. As agent adoption grows, programmes will need to measure not just privilege breadth but the compounded impact of chained actions, which is where classic IAM models are least mature.

For practitioners

• Define agent identity boundaries before deployment Document which AI agents can access which tools, datasets, and downstream services, then treat those boundaries as identity controls rather than application settings.
• Instrument agent sessions for multi-step behaviour Log prompts, tool calls, decision points, and downstream actions in one trace so analysts can reconstruct whether the agent stayed within intended scope.
• Map agent incidents to IAM and PAM controls Translate observed agent failures into access review, privilege assignment, and escalation controls so governance tracks real runtime behaviour instead of abstract model risk.
• Build low-noise detections for agent workflows Tune alerts to detect chained actions, unexpected tool combinations, and repeated execution patterns that indicate goal hijacking or cascading failure.
• Use OWASP Agentic Security Initiative outputs as a living control input Align internal control design with emerging agentic security guidance, then validate it against production telemetry and incident reviews.

Key takeaways

• Agentic AI security frameworks need to be built from real incidents, because theoretical models miss how threats repeat in production.
• Autonomous agent behaviour breaks assumptions embedded in access reviews, entitlement models, and static identity governance.
• Practitioners should treat agent identity, runtime telemetry, and accountability as one control plane, not separate discussions.

Key terms

• Agentic AI Security: Agentic AI security is the discipline of governing AI systems that can choose actions, use tools, and execute workflows at runtime. The control problem is not just model safety, but identity, privilege, telemetry, and accountability across the full action chain.
• Goal Hijacking: Goal hijacking happens when an AI agent’s original objective is redirected by malicious input, poisoned context, or unsafe tool use. In security terms, the threat is that the system keeps its legitimate access while its runtime intent changes, producing unauthorized outcomes.
• Identity Blast Radius: Identity blast radius is the practical extent of damage an identity can cause once it has access to tools, data, and downstream systems. For agentic systems, it expands quickly because one authorised action can trigger several additional actions without human review.

What's in the full article

Zenity's full article covers the operational detail this post intentionally leaves for the source:

• How Zenity translates real incident patterns into the OWASP Top 10 for Agentic Applications
• The researcher perspective on detection design, recall, and alert fatigue in enterprise AI security
• The role of community working groups, government conversations, and the Agentic Research Council
• Why the author believes incident sharing is necessary to make frameworks practical

👉 Zenity's full post covers the research journey, the OWASP Agentic Security Initiative, and the community work behind the framework.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM or identity security programme, it is worth exploring.