AI red teaming for agentic workflows: what changes for practitioners?

TL;DR: AI red teaming must evaluate entire agentic systems, not just isolated model endpoints, because tool use, APIs, data retrieval and workflow steps expand the attack surface, according to DeepKeep research. The shift makes reproducible replay, deterministic evidence and remediation-ready outputs central to enterprise AI assurance.

Editorial analysis by NHI Mgmt Group, based on content published by DeepKeep: “The AI Red Teaming Reality Check: How DeepKeep Delivers on OWASP”.

Key questions

Q: How should security teams test GenAI systems for prompt injection?

A: Test the full path, not just the chat box.

Q: Why do agentic AI incidents quickly become identity and privilege problems?

A: Because once an agent reaches credentials or inherited trust, the incident is no longer about model output.

Q: What are the signs that AI red teaming is too shallow?

A: The usual signs are narrow test cases, no coverage of live workflows, no retesting after model or prompt changes, and findings that are not linked to remediation.

Practitioner guidance

• Define the full agentic attack surface Map every tool, API, database and retrieval path the agent can reach so red teaming covers the entire workflow instead of the model endpoint alone.
• Log every step of each test run Capture prompts, responses, intermediate decisions and tool outputs so successful abuse paths can be replayed deterministically during remediation.
• Review delegated privileges before testing Inventory the permissions granted to agents and reduce overbroad access before you start assessing how far a malicious prompt can push them.

Bottom line: AI red teaming now has to evaluate what an agent can do across tools, APIs and workflows, not just what a model says in isolation.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →