Agentic AI red teaming: what security leaders need to prepare for

TL;DR: AI attacks are moving beyond prompt injection into agentic tools, memory, multimodal inputs, and supply chain paths, according to Mindgard, while studies show up to 97.14% jailbreak success and more than 73% of tested LLM apps vulnerable to prompt injection. Point-in-time reviews are no longer enough; continuous, hybrid testing has become the governing assumption.

Editorial analysis by NHI Mgmt Group, based on content published by Mindgard: “AI Red Teaming Trends: What Security Leaders Need to Prepare for Next”.

By the numbers:

• fully automated jailbreak agents have achieved success rates as high as 97.14% in one study.
• indirect prompt injection attacks succeed only 0.5%-8.5% of the time.
• attack success rate jumps from 13% for single-turn attacks to 92% for multi-turn conversations.

Key questions

Q: How should security teams test generative AI systems for real-world abuse?

A: They should test the full execution path, not just the model prompt.

Q: Why do agentic AI systems create more security risk than standard chatbots?

A: Agentic systems can turn model output into action, which means a bad instruction can affect code flow, tool use, and downstream state.

Q: What are the signs that voice AI red teaming is failing to keep pace?

A: Common warning signs are repeated unsafe completions, different behaviour after retraining, inconsistent handling of similar prompts, and security findings that reappear when conversation patterns change.

Practitioner guidance

• Map every agent permission and tool path Document which tools, APIs, memory stores, and external content sources each AI system can reach, then classify which actions are read-only, write-capable, or business-changing.
• Move from point-in-time tests to continuous red teaming Retest after prompt changes, retrieval changes, model changes, and workflow changes so new attack paths are found before they reach production.
• Add provenance controls to all inbound AI content Screen documents, web pages, images, and other retrieved material for hidden instructions, and separate trusted operator intent from untrusted source content.

Bottom line: AI red teaming is shifting from single-prompt testing to continuous assurance across tools, memory, retrieval, and delegated actions.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →