Topic starter
19/08/2026 3:36 pm
TL;DR: Most enterprise AI red teaming still tests a model once, against anticipated threats, before production, but ActiveFence argues that real incidents emerge later as systems drift, users adapt, and agents touch more of the stack than the lab can model. Continuous testing, runtime guardrails, and post-launch enforcement are now the governing controls, not optional extras.
Editorial analysis by NHI Mgmt Group, based on content published by ActiveFence: “What AI Red Teaming Looks Like Outside the Lab”.
By the numbers:
- In February, Cisco reported an average jailbreak success rate of 64% across models.
- Cisco also found that peak success rates for open-source models reached 93% during multi-turn attacks.
- A joint paper from OpenAI, Anthropic, and Google DeepMind found adaptive attacks succeeding more than 90% of the time against published model defenses.
Key questions
Q: What breaks when AI controls stop at pre-deployment testing?
A: Pre-deployment testing cannot stop a compliant model from making risky decisions in a live workflow or through connected tools.
Q: Why does runtime AI drift create more risk than a one-time red team result?
A: Because the risk comes from change.
Q: What are the signs that AI agent testing is too shallow?
A: If evaluation only checks refusal behaviour, overall benchmark scores, or end-to-end task success, it is probably missing the real attack surface.
Practitioner guidance
- Split pre-launch testing from runtime enforcement Use pre-launch red teaming to find obvious weaknesses, but define runtime guardrails, escalation paths, and monitoring as separate controls that must be live on day one.
- Add production drift to your testing scope Retest after model updates, workflow changes, new integrations, and user behaviour shifts so that post-launch change becomes part of the assurance cycle.
- Scope agent permissions as non-human identities Treat each AI agent as a delegated identity with explicit tool access, revocation criteria, and auditability for any API or workflow it can invoke.
Bottom line: Pre-launch AI red teaming can find known weaknesses, but it does not control how models behave after release when drift, integrations, and user behaviour change.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
19/08/2026 4:16 pm
Continuous AI red teaming is now a runtime control problem, not a lab exercise. A one-off adversarial test can expose known weaknesses, but it does not govern how the system behaves after model updates, workflow expansion, or user adaptation. That leaves a gap between validation and real-world assurance, which is where most AI incidents emerge. Practitioners should treat red teaming as a living control, not a launch gate.
A few things that frame the scale:
- 92% of organisations expose NHIs to third parties, raising concerns about supply chain security, according to the Ultimate Guide to NHIs.
A question worth separating out:
Q: How should teams govern AI agents that can reach APIs, events, and memory?
A: Teams should govern those agents as runtime identities, not as isolated integrations. That means enforcing policy at execution time, logging every tool and data access, and binding actions back to a clear initiating workflow or identity. If the control plane cannot show who acted, what they reached, and why, the programme does not have usable governance.
👉 Read our full editorial: AI red teaming fails when testing stops at pre-launch