Join our Newsletter — 33% off our NHI Course

AI agent workflows and fragile intent: are your tests keeping up?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 20739
Topic starter  

TL;DR: AI security testing has moved from model prompts to full agent workflows, where tools, APIs, memory, and multi-step interactions create a larger attack surface and make input or output filtering insufficient, according to Lasso Security. Coverage now has to be continuous, behavioral, and intent-aware because agents can drift, be redirected, and act outside their intended scope without obvious single-step failures.

Editorial analysis by NHI Mgmt Group, based on content published by Lasso Security: “AI Security Testing Has a Coverage Problem. Automated AI Red Teaming Fixes It.”.

Key questions

Q: How should security teams test AI agents after prompts, models, or tools change?

A: They should test AI agents every time a meaningful change occurs, not on a calendar alone.

Q: Why do AI agents create access risks that normal prompt filters do not solve?

A: AI agents combine language understanding with permissions, retrieval, and tool execution.

Q: What are the signs that an AI agent is being manipulated into unauthorized actions?

A: Warning signs include unexpected data retrieval, unusual tool calls, changes in agent decisions after hostile or extreme inputs, and outputs that reflect hidden instructions rather than the user’s request.

Practitioner guidance

  • Build a complete agent inventory Map every agent, the tools it can call, the APIs it can reach, the prompts it uses, and the data sources it depends on before testing begins.
  • Test multi-turn redirection paths Create adversarial conversations that span several exchanges to see whether the agent can be steered from refusal to compliance without an obvious rule break.
  • Version and review system prompts Treat system prompts as security-sensitive configuration, with change control, review, and regression testing whenever the prompt or its dependencies change.

Bottom line: AI security testing now has to follow agent workflows because tool use, memory, and multi-step execution create risks that prompt filtering cannot see.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 5 hours ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 5 months ago
Posts: 21367
 

AI security testing has become an identity problem, not just a model problem. Once an agent can select tools, call APIs, and carry context across steps, the control question shifts from what it outputs to what it is authorized to do. That is why agentic systems belong in the same governance conversation as non-human identities, because the failure mode is now execution, not content. Practitioners should treat workflow access as the real testing target.

A few things that frame the scale:

  • 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials, according to AI Agents: The New Attack Surface report.
  • Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.

A question worth separating out:

Q: What should organisations do when an AI agent’s scope keeps changing?

A: They should move from point-in-time review to continuous validation tied to lifecycle events such as model updates, prompt edits, and new tool connections. If the agent’s effective scope changes often, governance has to track the workflow as a living access boundary, not a fixed application setting.

👉 Read our full editorial: AI security testing now needs coverage across agent workflows



   
ReplyQuote
(@mr-nhi)
Member Moderator
Joined: 5 months ago
Posts: 21367
 

AI security testing has crossed from content safety into execution governance. The article is right to separate model-level filtering from agent workflow coverage, because the dangerous outcome is no longer just a bad answer. When an agent can call tools and act across systems, the question becomes whether its execution path stays inside the intended boundary. That shifts security testing into identity, access, and behaviour control, not just prompt evaluation. Practitioners should treat workflow execution as the real unit of risk.

A few things that frame the scale:

A question worth separating out:

Q: What should teams do first when red-teaming an AI agent?

A: Start by writing a scope document that inventories the agent, its tools, data sources, memory, and blast radius. Without that baseline, testing becomes a sequence of prompts with no security meaning. Scope turns a loose exercise into a controlled assessment that can be repeated, audited, and compared across harnesses.

👉 Read our full editorial: AI security testing now needs coverage across agent workflows


This post was modified 5 hours ago by NHI Mgmt Group

   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.