Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

AI agent security testing: are traditional controls keeping up?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 5324
Topic starter  

TL;DR: AI agents create a security problem that traditional tooling cannot fully cover because the attack surface shifts into prompt manipulation, data poisoning, tool misuse, and weak observability, according to ZioSec, with 28 million AI-driven cyberattacks projected in 2025 and 87% of organizations targeted in the last year. Autonomous agent governance now depends on testing reasoning paths, tool permissions, and logging, not just code and infrastructure.

NHIMG editorial — based on content published by ZioSec: Break Your Own AI Agent, Part 1

By the numbers:

Questions worth separating out

Q: How should security teams test AI agents for prompt injection risk?

A: Use adversarial inputs that place conflicting instructions in prompts, retrieved documents, and memory.

Q: Why do AI agents complicate least-privilege design?

A: AI agents complicate least privilege because they can choose tools and sequence actions at runtime, often in ways the original designer did not anticipate.

Q: What do organisations get wrong about AI agent logging?

A: They log API activity but fail to capture the full decision trace.

Practitioner guidance

  • Test prompt and retrieval boundaries Run adversarial cases that mix benign and malicious instructions in user prompts, RAG sources, and memory inputs.
  • Constrain tool permissions to business-safe actions Map every agent tool to a specific action class, then remove anything that would let one session read, write, or trigger actions outside its stated use case.
  • Build agent-level telemetry before rollout Capture prompts, retrieved context, tool calls, and outputs in one trace so investigators can reconstruct the decision path after an incident.

What's in the full article

ZioSec's full blog post covers the operational detail this post intentionally leaves for the source:

  • A fuller breakdown of prompt injection, data poisoning, and tool misuse patterns in agentic systems.
  • Specific examples of how attackers abuse authorised agent tools without breaking traditional authentication controls.
  • More detail on logging and monitoring gaps that make agent forensics difficult after an incident.
  • The article's forward-looking discussion of why builders need internal red teaming as part of the development lifecycle.

👉 Read ZioSec's analysis of proactive security testing for AI agents →

AI agent security testing: are traditional controls keeping up?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 1 month ago
Posts: 3993
 

AI agents turn access control into behaviour control. Traditional IAM assumes the subject will request, receive, and use access in predictable ways. That assumption fails when the actor can choose actions at runtime, combine tools dynamically, and continue operating without human approval. The implication is that governance can no longer stop at entitlement design, because runtime behaviour becomes the real security boundary.

A few things that frame the scale:

  • 92% agree governing AI agents is critical to enterprise security, yet only 44% have implemented any policies to do so, according to AI Agents: The New Attack Surface report.
  • Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.

A question worth separating out:

Q: Who should own AI agent governance in an enterprise?

A: Ownership should sit across security, IAM, application, and platform teams, because agent governance spans entitlement design, runtime behaviour, data access, and lifecycle control. If ownership is left only to developers, the programme usually misses offboarding, monitoring, and access review discipline.

👉 Read our full editorial: AI agent security testing reveals the limits of traditional controls



   
ReplyQuote
Share: