Topic starter
02/08/2026 12:09 pm
TL;DR: AI red teaming must move from one-off validation to continuous testing because real-world AI flaws are increasingly triggered by prompt manipulation, data poisoning, model extraction, and tool abuse, according to Akto’s guide and cited research. Probabilistic behaviour changes the attack surface, so resilience depends on repeatable adversarial testing, not standard penetration testing alone.
Editorial analysis by NHI Mgmt Group, based on content published by Akto: “AI Red Teaming: How to Continuously Test and Secure Agentic AI Systems”.
By the numbers:
- Attacks with roleplay cost 89.6% success, according to Akto.
Key questions
Q: Why do agentic AI systems require continuous red teaming?
A: Agentic systems change when models are retrained, prompts are updated, retrieval sources shift, or new integrations are added.
Q: Why do agentic AI systems need continuous testing instead of periodic pen tests?
A: Agentic systems are probabilistic, so the same input may behave differently across sessions, model states, and tool contexts.
Q: What do security teams get wrong about prompt injection in agentic workflows?
A: They often treat prompt injection as an input validation problem only.
Practitioner guidance
- Shift to continuous AI red teaming Retest agentic systems after model updates, prompt changes, new tools, and integration changes so assurance follows the lifecycle rather than the release date.
- Separate manual discovery from automated regression Use human testers to find novel jailbreaks, roleplay attacks, and logic traps, then convert validated cases into automated checks that run on every build and deployment.
- Test the full prompt-to-tool chain Trace how untrusted input moves from user text into retrieval, tool calls, and external actions, and verify that each handoff enforces its own trust boundary.
Bottom line: Agentic AI testing fails when teams treat model behaviour as stable, because context, tools, and prompts can change the outcome of the same input.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
02/08/2026 1:13 pm
Continuous testing is now the correct control model for agentic AI. A one-time red team cannot keep pace with probabilistic behaviour, model updates, and changing tool chains. The article makes the underlying point clearly: if the system can change its response from one run to the next, assurance has to move with it. For practitioners, the operational question is not whether red teaming exists, but whether it is embedded into the AI lifecycle.
A few things that frame the scale:
- Gartner predicts that by the end of 2026, 40% of enterprise apps will feature task-specific AI agents.
A question worth separating out:
Q: What happens when an agentic system is tested only once before release?
A: A one-time test can give false confidence because the model, prompts, tools, and connected data sources keep changing after launch. A control that worked in one session may fail in another. That leaves organisations exposed to regressions, repeated jailbreaks, and tool abuse that no longer match the original test conditions.
👉 Read our full editorial: AI red teaming for agentic systems needs continuous testing