Security teams should test the full AI stack as one attack surface, not as isolated components. Use probes that cover prompt injection, agent goal hijacking, tool poisoning, privilege escalation, and secret exposure across LLMs, autonomous agents, and MCP servers. The goal is to find chained failures before an attacker can move from model compromise to unauthorized actions.
Testing the Full Agentic AI Chain, Not Just the Model
Red teaming agentic AI works best when security teams treat the model, orchestration layer, and MCP services as one connected control plane. A model-only test can miss the more dangerous failure: an apparently safe response that still drives a tool call, a workflow handoff, or a privileged retrieval action. For that reason, the test plan should include prompt manipulation, tool invocation abuse, session boundary confusion, and secret exposure across the full execution path.
That matters because agentic systems fail at the seams. The model may appear constrained, while the orchestrator silently expands context, retries actions, or passes credentials to a downstream service. The MCP layer is especially important where tool descriptions, resource exposure, or server-side assumptions create a path from language output to real action. In practice, security teams often discover the highest-impact weakness only after a benign-looking prompt has already crossed from dialogue into execution.
External guidance from the OWASP Top 10 for Agentic Applications 2026 is useful here because it frames the agent as an application with attackable boundaries, not just a model prompt.
How Red Team Scenarios Should Traverse Model, Orchestration, and MCP Layers
Effective red teaming starts with a clear chain: what the model can be induced to say, what the orchestrator can be induced to do, and what the MCP layer can be induced to expose or execute. The objective is not to break each layer in isolation, but to test whether one layer can be used to influence the next. That means scenario design should move from content injection to decision manipulation to action execution.
A practical test set usually includes multiple classes of probe:
- Prompt injection that attempts to override system intent, policy, or task priority.
- Goal hijacking that changes the agent’s objective without obviously malicious wording.
- Tool poisoning that manipulates tool descriptions, retrieved instructions, or resource metadata.
- Privilege escalation attempts that target workflow tokens, delegated permissions, or implied trust.
- Secret exposure checks that look for tokens, keys, or sensitive context leaking into prompts, logs, or tool calls.
The orchestration layer deserves special attention because it often makes the actual security decision. It decides what context to append, whether to retry, what tool to invoke, and whether a result is trusted enough to continue. If that layer does not enforce strict action boundaries, the model becomes a suggestion engine that can still trigger real-world side effects. The MCP layer should be tested as a trust boundary too, because a well-formed tool interface can still become dangerous if it accepts overbroad parameters, weakly scoped resources, or ambiguous permission assumptions.
For adversarial behaviour patterns, the MITRE ATLAS adversarial AI threat matrix helps structure probes around realistic attack patterns rather than abstract failure modes.
Where the guidance breaks down is in highly bespoke agent workflows that rely on custom memory, chained tools, or human approval steps, because those flows often need scenario-specific tests rather than a generic red-team script.
Where Agentic Red Teaming Gets Harder: Memory, Delegation, and Tool Trust
Tighter testing often increases execution complexity, requiring teams to balance realism against the risk of accidentally granting the system more trust than it should have. That tradeoff becomes visible in memory-heavy agents, delegated workflows, and MCP environments where the agent can carry forward state from one interaction to the next.
One common edge case is a system that looks safe in a single-turn test but fails when the same malicious instruction is preserved in memory or reintroduced through retrieval. Another is a workflow where the model is constrained, yet the orchestrator accepts tool output as authoritative without re-checking the source of the instruction. In MCP deployments, the important question is not only whether a tool exists, but whether the agent can infer enough structure from tool metadata to misuse it.
There is also a genuine consensus gap in the industry around how much autonomy is acceptable in test environments. Some teams simulate full tool access to discover worst-case chains; others prefer staged access to avoid contaminating shared systems. Both approaches are defensible, but the safer choice depends on whether the goal is exploitation discovery, control validation, or operational readiness. For threat modeling of these agent-specific trust chains, the CSA MAESTRO agentic AI threat modeling framework is a relevant companion reference.
Risk and Threat Considerations
Agentic AI systems create compound risk because a weakness in the model can become a control failure in orchestration and then a real action in MCP-connected services. The material threat is not just bad output, but abuse of delegated authority, unsafe tool invocation, and credential or data exposure through chained execution.
Failure mechanism: An attacker uses prompt injection, goal hijacking, or tool manipulation to influence the model, then relies on the orchestrator to preserve or amplify that instruction, and finally abuses the MCP layer to execute an unauthorized action or retrieve sensitive material. The attack works when trust is transferred across layers without re-validation at each boundary.
Impact: The result can be unauthorized tool use, leakage of secrets or sensitive context, corrupted workflow decisions, or lateral movement into connected systems through an agent that was never meant to have that level of autonomy.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, MITRE ATLAS and CSA MAESTRO address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A1 — Prompt Injection and Instruction Manipulation | Directly covers attacker manipulation of agent instructions and task steering. |
| A3 — Tool / Function Abuse | Matches abuse of agent tools, delegated actions, and unsafe executions. | |
| A5 — Memory and State Poisoning | Relevant to persistence of malicious instructions across agent state and context. | |
| Recommendation — Test for instruction override paths that can redirect agent decisions or tool use. Probe tool boundaries for unauthorized actions, overbroad calls, and unsafe delegation. Validate whether malicious state persists and alters later agent behaviour. | ||
| MITRE ATLAS | AML.TA0001 — Reconnaissance | Useful for structuring adversarial probes against AI systems and exposed surfaces. |
| AML.TA0002 — Initial Access | Fits prompt injection and other entry techniques used to gain influence over AI systems. | |
| AML.TA0006 — Evasion | Covers techniques that hide malicious intent inside benign-looking interactions. | |
| Recommendation — Map agent surfaces to adversarial test cases that reveal exposed trust boundaries. Hunt for entry paths that let attackers influence prompts, tools, or orchestration. Test whether malicious instructions can evade policy and safety checks. | ||
| CSA MAESTRO | MAESTRO-TRM-01 — Threat Modeling | Agentic red teaming depends on modelling end-to-end trust and execution chains. |
| Recommendation — Model model-to-orchestrator-to-tool trust chains before executing red-team scenarios. | ||
| NIST AI RMF | MAP — Map | Supports identifying AI system context, scope, and risk surfaces before testing. |
| MEASURE — Measure | Fits evaluation of agent behaviour, failure modes, and control effectiveness. | |
| MANAGE — Manage | Relevant to governance of residual AI risk and escalation of unsafe autonomy. | |
| Recommendation — Map model, orchestration, and MCP dependencies before defining adversarial test coverage. Measure chained failure rates across prompts, tools, and delegated actions. Use governance gates to limit autonomy where tests show unsafe escalation paths. | ||
Practitioner Guidance
What to prioritise: Start with chained failure paths, not isolated jailbreaks. The most valuable red-team findings are usually the ones that show how a harmless-seeming model response becomes an orchestrated action or privileged tool call.
What to verify: Confirm that each layer independently validates intent, scope, and authority before passing work to the next layer. If the orchestrator trusts model output by default, or the MCP server trusts the agent by default, the test should be treated as incomplete even if the model itself appears resistant.
What practitioners underestimate: The hardest problems often sit in the metadata and state-handling layers, not in the prompt itself. Teams frequently overfocus on jailbreak wording and under-test context reuse, retry logic, hidden memory, and tool descriptions that can quietly reshape agent behaviour.
Practitioner takeaway: Red teaming agentic AI is most useful when it proves whether the system can turn a language-level compromise into an authority-level failure, because that is where the real blast radius begins.
Related resources from NHI Mgmt Group
- How should security teams red team agentic AI systems differently from standalone chatbots?
- How should security teams govern AI agent orchestration across multiple systems?
- How should security teams red team non-deterministic AI systems?
- How should security teams evaluate AI red teaming vendors for agentic systems?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 7, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
