Notifications
Clear all
Topic starter
06/09/2026 6:21 pm
TL;DR: AI red teaming targets deployed LLM applications, not models in isolation, and Novee argues that prompt injection, tool misuse, and retrieval poisoning must be tested continuously because application changes quickly alter the attack surface. For IAM and NHI teams, the key shift is treating connected tools and agentic workflows as governed identities and access paths, not just model features.
NHIMG editorial — based on content published by Novee: How Modern Security Teams Use AI Red Teaming
By the numbers:
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes.
Questions worth separating out
Q: What breaks when AI security stops at model scanning?
A: Model scanning helps identify tampering and unsafe dependencies before deployment, but it does not address runtime misuse.
Q: When does AI red teaming need to move from periodic testing to continuous testing?
A: Continuous testing becomes necessary when model updates, new integrations, or prompt changes can alter agent behaviour without a code rewrite.
Q: What are the signs that prompt injection is becoming a governance problem?
A: Warning signs include rising jailbreak success rates, tools being called in ways the business did not intend, and changes in behaviour after prompt or connector updates.
Practitioner guidance
- Map every AI application to its real attack surface Inventory retrieval sources, APIs, tool connectors, system prompts, and downstream systems before approval so red team coverage matches the deployed architecture.
- Treat AI tool access as governed privilege Assign the minimum possible permissions to service accounts and agent workflows, then define explicit revocation and rotation paths for every connected secret.
- Run adversarial tests on every material change Re-test after prompt edits, model swaps, new connectors, or tool permission changes because each of those can reopen an attack path that was previously closed.
What's in the full article
Novee's full research covers the operational detail this post intentionally leaves for the source:
- Step-by-step red teaming workflows for LLM-powered applications, including scoping, probing, validating, and reporting.
- Examples of prompt injection, jailbreak, and tool-abuse payloads that can be adapted to different application architectures.
- How automation and human operators combine to test agentic workflows and track regression over time.
- The source article's practical mapping of AI red teaming to OWASP LLM application risks and enterprise deployment patterns.
👉 Read Novee’s analysis of how modern security teams use AI red teaming →
AI red teaming and LLM apps: are your controls keeping up?
Explore further
06/09/2026 6:22 pm
AI red teaming has become an identity and privilege problem, not only a model-safety problem. Once an LLM can call tools, query data, or act on behalf of users, its permissions behave like non-human identity entitlements. That means the security question is no longer only whether the model can be fooled, but whether the surrounding access model is narrow enough to limit damage. IAM and PAM teams should treat agentic workflows as governed identities with explicit scope, lifecycle, and revocation rules.
A question worth separating out:
Q: How should security teams govern AI models that can call tools and access data?
A: Security teams should govern AI models as non-human identities with named owners, limited scope, short-lived credentials, and continuous authorization. The critical shift is to treat every tool call, data read, and update path as a privileged action that can be logged, revalidated, and revoked. Without that discipline, model risk becomes identity risk.
👉 Read our full editorial: AI red teaming is now a governance issue for LLM applications
