Red teaming agentic AI workflows is now an execution-layer problem

At a glance

What this is: Lakera’s research argues that red teaming agentic AI must evaluate the full execution layer, because prompts alone do not reveal how risk emerges across tools, memory, handoffs, and multi-step workflows.

Why it matters: IAM and security teams need to govern agentic systems as active identity-bearing workflows, not static models, because access decisions, tool use, and execution paths can create compound risk.

Context

Agentic AI red teaming is the practice of testing how an AI system behaves when it can call tools, carry state forward, and move through multiple execution steps. In that setting, the security question is no longer only whether the model answers safely, but whether the workflow behaves safely under pressure.

Lakera’s article focuses on the execution layer because failures can emerge in tool boundaries, intermediate state, memory, and multi-agent handoffs even when the base model appears well behaved. That makes the identity and access problem broader than prompt injection alone, especially for teams governing AI systems that act inside enterprise workflows.

The article’s starting position is typical for agentic systems now moving toward production: the system is not broken at one point, but brittle across the chain of decision and action.

Key questions

Q: What breaks when agentic AI red teaming stays at single prompts?

A: Single-prompt testing misses stateful failures that emerge only after tool calls, memory updates, and follow-on decisions interact. In agentic systems, the risky behaviour often appears in the second or third step, not the first prompt. Teams need scenario-based testing that follows the entire workflow, because the security issue is the sequence, not only the input.

Q: Why do agent workflows create more governance risk than standalone models?

A: Agent workflows create more governance risk because they combine reasoning with action. Once tools, memory, and handoffs are involved, a failure in one step can affect later decisions or external actions even if the base model seems safe. Governance must therefore cover execution paths, not just outputs.

Q: How do security teams know if AI red teaming is working?

A: AI red teaming is working when testing finds real prompt injection paths, over-scoped integrations, and policy gaps before attackers do, and when fixes are re-tested successfully after model or workflow changes. The strongest signal is repeatable reduction in exposed authority, not a lower number of red-team findings on its own.

Q: What should teams do after an agent red team run finds workflow weaknesses?

A: They should fix the weak execution points, then rerun the same scenarios against the updated system. The goal is to prove that guardrails, output validation, or prompt changes altered behaviour across the full workflow, not only at the model boundary.

Technical breakdown

Why model-level checks miss agentic workflow risk

Model-level checks evaluate a single interaction, but agentic systems behave as workflows. Once a system can retrieve data, invoke tools, and carry state across steps, the attack surface includes not only prompts but also tool outputs, intermediate decisions, memory, and handoffs. A failure in one stage can shape what the next stage does, which means the weakness may only appear when the system is exercised end to end. That is why isolated testing can overstate resilience: the model may refuse an unsafe prompt while the workflow still reaches the same harmful outcome through another path.

Practical implication: Test the whole agent flow, not just the base model response.

How risk propagates across multi-step execution

Risk propagation is the way an early weakness influences later behaviour. In an agentic workflow, a manipulated input or unsafe tool response can alter reasoning, change retrieval results, or steer the final action even if later components are individually defensive. Lakera’s framing shows why evaluation has to measure not only whether a step fails, but whether that failure is amplified, filtered, or transformed as the workflow continues. This is especially important when the same attack can produce different outcomes across runs, because variance itself becomes a governance signal.

Practical implication: Measure where a failure originates and how far it travels through the workflow.

Why structured scoring matters in agent red teaming

Structured scoring turns red teaming from a one-off test into a repeatable control. By normalizing risk scores and attack success rates, teams can compare runs over time, compare scenarios, and see whether a mitigation actually reduced exposure. That matters for agentic systems because the same workflow can behave differently depending on input path, tool availability, or state carried from previous steps. A consistent evaluation format gives security and engineering teams a shared way to decide whether the system is becoming safer or merely producing different failure patterns.

Practical implication: Use normalized scoring to compare agent behaviour before and after mitigations.

Threat narrative

Attacker objective: The attacker wants the agent to carry a manipulated instruction or input through execution until it produces an unsafe outcome.

1. Entry occurs when an adversarial prompt or indirect input reaches a user-facing step or a data source the agent consumes.
2. Escalation occurs when that initial manipulation influences tool selection, intermediate reasoning, or memory, allowing the attack to shape later decisions.
3. Impact follows when the workflow produces harmful content, data exposure, or unsafe recommendations through the full execution chain.

Breaches seen in the wild

• NVIDIA code-signing certificates stolen 2022: Lapsus$ leaked two NVIDIA code-signing certificates; Windows still trusted them for drivers and attackers signed Cobalt Strike and Mimikatz with them.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Execution-layer governance is now the right unit of control for agentic AI. The article’s central point is that prompt checks do not describe the real security boundary once an agent can call tools, retain state, and hand work across steps. That moves the governance question from what the model said to what the workflow did. Practitioners should treat the execution path as the unit of risk.

Prompt-only assurance is an incomplete control model for agentic systems. A system can reject obvious harmful prompts and still fail through indirect inputs, tool responses, or multi-step state transitions. That is not a minor gap in testing discipline. It means the traditional assumption that a single decision point captures the main risk no longer holds for agentic workflows.

Risk propagation is the distinctive failure mode in agentic AI. Lakera’s framing shows that a weakness does not need to succeed at the first step to become operationally meaningful. It can be amplified, attenuated, or transformed as the workflow progresses, which makes end-to-end evaluation more useful than component testing alone. The practitioner conclusion is to govern behaviour across the chain, not at a single checkpoint.

Agentic AI red teaming should be measured like a repeatable control, not a research exercise. Structured findings, normalized risk scores, and attack success rates make the output comparable across releases and scenarios. That creates an evidence base for whether mitigation actually changes behaviour. For IAM and security teams, the practical takeaway is that agent governance needs observable outcomes, not just policy language.

Memory, tools, and handoffs are identity-adjacent control points, even when the article does not frame them that way. Once an agent can invoke external actions and preserve context, security can no longer rely on model behaviour as a proxy for authorisation safety. The control boundary shifts to what the system can access, in what sequence, and under what execution conditions. Practitioners need to design governance around those runtime boundaries.

What this signals

Execution-layer visibility will become a baseline expectation for agent governance. Teams that only observe prompts will miss the places where tool calls, memory, and handoffs change the outcome. The practical shift is toward instrumenting the workflow itself so that access, action, and propagation can be reviewed together.

Agent red teaming needs to look more like operational assurance than model evaluation. A security programme for autonomous behaviour has to show whether a system is safer after a change, not just whether a test passed once. That pushes governance toward repeatable scenarios, comparable results, and release-stage evidence.

Workflow brittleness is the signal to watch. When the same attack sometimes succeeds and sometimes fails, the system is not stable enough for trust decisions. For practitioners, that variability is a warning that control design must account for execution context, not just policy intent.

For practitioners

• Map the full agent workflow Document user inputs, tool boundaries, intermediate state, memory, and handoff points before you treat the system as testable or governable.
• Run end-to-end red team scenarios Exercise the agent across complete execution paths, including indirect inputs and tool responses, so you can observe how failures emerge downstream.
• Track risk propagation, not just attack success Compare where a failure starts, where it spreads, and where it is filtered so you can distinguish local weakness from workflow-wide brittleness.
• Use normalized scores as a release gate Treat attack success rate and normalized risk score as repeatable measures for deciding whether a mitigation changed the workflow outcome.
• Re-test after each mitigation Re-run the same scenarios after guardrails, prompt hardening, or output validation changes so you can prove whether the execution layer became more resilient.

Key takeaways

• Agentic AI red teaming has to evaluate the full execution layer because workflow behaviour can create risk that prompt-only checks never see.
• Tool calls, memory, handoffs, and multi-step execution are the points where failures propagate and compound across an agentic system.
• The practical response is to test end to end, measure repeatably, and treat workflow behaviour as the control surface that matters.

Key terms

• Agentic workflow: An agentic workflow is a sequence of tasks executed by an AI agent with some level of tool access and decision authority. In security terms, the workflow matters because it can span multiple systems, identities, and permissions, which makes attribution and revocation harder than with ordinary automation.
• Risk Propagation: Risk propagation is the way a failure in one part of a system influences later steps, outputs, or actions. For agentic systems, it describes how an unsafe input, tool response, or intermediate state can spread across the workflow and create a larger operational impact.
• Attack Success Rate: Attack success rate is the percentage of test attempts in which an injected adversarial scenario achieved its intended effect. For agentic systems, it helps distinguish a one-off anomaly from a repeatable workflow weakness that requires governance action.
• Normalized Risk Score: A normalized risk score is a standardised measure used to compare how vulnerable an agent is across different scenarios or releases. It becomes useful only when teams use it to drive decisions about access, tool scope, guardrails, and monitoring.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.