Topic starter
06/09/2026 6:21 pm
TL;DR: AI red teaming targets deployed LLM applications, not models in isolation, and Novee argues that prompt injection, tool misuse, and retrieval poisoning must be tested continuously because application changes quickly alter the attack surface. For IAM and NHI teams, the key shift is treating connected tools and agentic workflows as governed identities and access paths, not just model features.
Editorial analysis by NHI Mgmt Group, based on content published by Novee: “How Modern Security Teams Use AI Red Teaming”.
Key questions
Q: What breaks when an LLM application is tested only as a model?
A: Model-only testing misses the real attack surface created by retrieval, tool access, and business logic.
Q: Why do connected tools turn prompt injection into a governance problem?
A: Because the injected instruction is no longer just text.
Q: How do security teams know if AI red teaming is working?
A: AI red teaming is working when testing finds real prompt injection paths, over-scoped integrations, and policy gaps before attackers do, and when fixes are re-tested successfully after model or workflow changes.
Practitioner guidance
- Map every connected tool and data source Inventory retrieval pipelines, API integrations, file stores, and workflow steps that an LLM application can reach, then classify which actions are read-only, write-capable, or externally visible.
- Limit delegated authority for agentic workflows Assign the narrowest feasible tool permissions, separate high-risk actions from low-risk ones, and require explicit approval for actions that alter records, send messages, or move data.
- Test for indirect prompt injection Include malicious instructions embedded in retrieved documents, web pages, and API responses so you can see whether the application treats untrusted content as commands.
Bottom line: LLM red teaming has to examine the deployed application, because the real exposure comes from connected tools, retrieval, and runtime authority, not from model output alone.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
06/09/2026 6:22 pm
LLM red teaming has become an access-governance discipline, not just an AI safety exercise. The article is right to separate model output testing from application testing, because the real risk appears when a model is connected to tools, retrieval, and business logic. That makes the control question one of delegated authority, not simply unsafe text generation. Practitioners should govern the application like any other privilege-bearing system.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: How should organisations govern agentic AI in observability workflows?
A: Treat AI as a bounded operator, not an all-purpose controller. Give it narrow permissions, require audit trails, and restrict it to approved actions such as validation, suggestion, or rollback of pre-defined changes. If the telemetry path is fragmented, fix that first, because agentic automation amplifies whatever control model already exists.
👉 Read our full editorial: AI red teaming is now a governance issue for LLM applications