Notifications
Clear all
Topic starter
24/06/2026 8:43 pm
TL;DR: AI agent evaluations can show expected behaviour, but live attacks reveal how tools, prompts, and runtime access combine to create failure paths that tests miss, according to ZioSec. The real issue is that agent governance is still being treated like static software testing when it needs identity-aware attack validation.
NHIMG editorial — based on content published by ZioSec: AI Agents: Evaluations Versus Attacks
Questions worth separating out
Q: How should security teams test AI agents beyond standard evaluations?
A: Security teams should combine evaluations with adversarial attack testing that manipulates prompts, tool calls, and runtime context.
Q: Why do AI agent attacks reveal more risk than evaluations alone?
A: Because attacks simulate hostile conditions that evaluations usually exclude.
Q: What do security teams get wrong about AI agent governance?
A: They often separate model testing from identity governance, even though the two failure modes are linked.
Practitioner guidance
- Run adversarial tests on agent identity paths Test how prompts, tool requests, and external data sources change agent behaviour under attack.
- Inventory every tool and credential an agent can reach Map each AI agent to the exact APIs, databases, and secrets it can access at runtime.
- Treat evaluations as one control, not the control Use evaluations to measure intended behaviour, then pair them with live attack campaigns that probe for privilege misuse, prompt injection, and unintended action chaining.
What's in the full article
ZioSec's full blog post covers the operational detail this post intentionally leaves for the source:
- How the team structures live attack campaigns against AI agents and what inputs they target.
- Examples of the kinds of agent failure patterns developers should look for during security review.
- The operational difference between evaluating model behaviour and testing runtime abuse paths.
- How a security architect frames offensive testing for teams building agentic workflows.
👉 Read ZioSec's analysis of AI agent evaluations versus attacks →
AI agent evaluations versus attacks: are your controls enough?
Explore further
25/06/2026 5:28 am
Evaluations prove intent, attacks prove exposure. AI agent evaluations are useful for measuring expected behaviour, but they do not prove that the agent is safe once adversaries can manipulate prompts, tools, or context. That distinction matters because identity security fails at the point where runtime trust is abused, not where a benchmark is passed. Practitioners should treat live attack testing as the control that exposes whether the agent’s access model is actually defensible.
A few things that frame the scale:
- The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to The State of Secrets in AppSec.
- Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap, according to The State of Secrets in AppSec.
A question worth separating out:
Q: How do organisations know if agent access controls are actually working?
A: They know only when controls are tested against hostile behaviour, not when the agent merely passes a benchmark. Look for evidence that tool access is limited, sensitive actions require explicit boundaries, and revocation works quickly when behaviour changes. If attack testing can still drive unauthorised actions, the controls are not effective enough.
👉 Read our full editorial: AI agent evaluations versus attacks expose a governance gap
