Notifications
Clear all
Topic starter
02/08/2026 11:32 am
TL;DR: AI red teaming for agentic systems must move beyond point-in-time testing because prompt injection, jailbreaks, tool misuse, and cross-agent contamination can change system behaviour at runtime, according to Akto’s analysis. Continuous adversarial validation, threat modeling, and regression testing are now the only defensible way to govern AI security posture at scale.
NHIMG editorial — based on content published by Akto: AI Red Teaming Best Practices for Agentic AI Security
Questions worth separating out
Q: What breaks when AI security is handled only at launch time?
A: Controls go stale as the system changes.
Q: Why do agentic AI systems complicate existing IAM and PAM controls?
A: They complicate them because IAM and PAM were built around stable identities, human-paced approvals, and entitlements that are reviewed after use.
Q: How do security teams know if AI red teaming is working?
A: AI red teaming is working when testing finds real prompt injection paths, over-scoped integrations, and policy gaps before attackers do, and when fixes are re-tested successfully after model or workflow changes.
Practitioner guidance
- Map agent authority before testing begins Document every prompt source, memory store, tool connector, and external API the agent can reach.
- Convert every exploit into a regression case Capture successful prompt injections, jailbreaks, and tool misuse chains as automated tests.
- Restrict agent tool permissions to task scope Remove unnecessary access to code execution, file systems, and external systems before production deployment.
What's in the full article
Akto's full blog covers the operational detail this post intentionally leaves for the source:
- Step-by-step red teaming workflow for planning, testing, reporting, and retesting AI systems.
- Practical examples of prompt fuzzing, jailbreak attempts, and tool abuse simulations across agentic workflows.
- Guidance for building continuous AI security posture management into CI/CD and runtime operations.
- Framework mapping details for OWASP, MITRE ATLAS, and NIST AI RMF in AI red team programmes.
👉 Read Akto's analysis of AI red teaming best practices for agentic AI security →
Agentic AI red teaming best practices: are your controls keeping up?
Explore further
02/08/2026 12:47 pm
AI red teaming is becoming a control assurance function, not a specialist exercise. Once AI systems can call tools and trigger downstream actions, the question is no longer whether a model generates bad text. The question is whether the system can be manipulated into taking harmful actions inside a live environment. That shifts red teaming from a periodic assessment into a governance mechanism for runtime behaviour, especially where the agent has access to credentials, connectors, or business systems. Practitioners should align testing with AI RMF GOVERN and MEASURE outcomes, not just model quality checks.
A question worth separating out:
Q: What should teams do when an agent can access sensitive tools or data?
A: They should reduce scope before production use, separate high-risk tools from low-risk ones, and require explicit approval for any action with external impact. If the workflow cannot be safely constrained, the access model is too permissive for an agentic system.
👉 Read our full editorial: AI red teaming for agentic systems needs continuous validation
