Should organisations combine AI red teaming with IAM and PAM reviews?

Why AI red teaming should include IAM and PAM reviews

ai red teaming is most useful when it tests the system the way an attacker or a badly governed user flow would use it, not just how the model answers prompts. If an AI can trigger tools, read data, or take action on behalf of a user, then access approval, privilege boundaries, session handling, and revocation are part of the same attack surface. That is why red teaming AI agents for identity abuse is a practical companion to model testing.

Teams often discover that the model is not the only weak point. The higher-risk failure is usually an overpowered delegated identity, a long-lived secret, or an approval path that lets the agent act more broadly than intended. Reviews that stop at prompt safety can miss the actual blast radius, while privileged access management and service account security help anchor the discussion in the permissions the AI can really use.

Practically, the question is whether the AI can be tricked into doing something that a human would not have been allowed to do, or whether it can do the right thing with the wrong scope of authority. If the answer is yes, then red teaming should examine the permissions model, not only the prompt chain. That includes tool invocation rights, data access boundaries, emergency access paths, and whether revocation actually stops the system from continuing to act.

What a combined test should examine

A combined exercise should follow the action path, from prompt to tool call to data access to downstream effect. That means checking whether the AI can request elevated actions, inherit a privileged session, reuse a token beyond its intended context, or bypass a human approval step. For cloud and enterprise environments, cloud PAM and CIEM help surface where effective permissions are wider than the role description suggests.

The review should also test whether revocation is real, not theoretical. If a secret is rotated, a role is removed, or a session is closed, the AI should lose the ability to continue the action path. That is especially important for long-running agents and integrations that keep cached credentials or maintain indirect access through service accounts. The issue is not only whether the initial control exists, but whether it actually constrains the full runtime behaviour.

One useful way to frame the exercise is to ask what would happen if the agent were impersonated, manipulated, or misdirected. If the answer includes access to production data, sensitive workflows, or administrative functions, the red team should validate the approval logic, logging, and session containment around that access path. Those controls are often what turn a prompt issue into a real security incident.

Where organisations usually find the gap

The most common gap is a mismatch between the model test scope and the operational scope. A team may test for harmful text output, but the actual risk comes from delegated identity, standing privilege, or poorly bounded access to tools and records. That is why AI red teaming should be paired with identity and privilege checks whenever the AI can take actions, not just generate content.

Another frequent gap is assuming that logging alone provides control. Logs are necessary, but they do not reduce authority. If an AI agent can still invoke a privileged action after the model has been constrained, then the control failure sits in the IAM or PAM layer, not the model layer. The combined review should therefore ask not only “did we detect it?” but also “could the agent have done it at all?”

Risk and Threat Considerations

When AI systems are allowed to act through credentials or delegated approvals, the main risk is overreach: the model may be safe enough, while the account behind it is not. That creates a path for prompt injection, tool misuse, or confused-deputy behaviour to turn a conversational flaw into real access, data exposure, or destructive change.

Failure mechanism: The attacker manipulates the AI’s instructions or inputs so the system uses a legitimate identity, session, or token to perform an action outside the intended business context. Weak approval design, excessive privilege, or poor revocation lets the action succeed even when the prompt itself looks benign.

Impact: The result can be unauthorized data access, privilege escalation, fraudulent workflow execution, or persistence through reused credentials and sessions. In practical terms, the blast radius is determined less by the model’s wording and more by the authority attached to the agent.

Practitioner Guidance

What to prioritise: Start with the highest-value actions the AI can trigger, then trace the exact identity, session, and privilege path behind each one. If the path reaches production data, finance, customer records, or admin functions, that flow belongs in the red-team scope.

What to verify: Confirm that each tool call has a clear owner, a bounded scope, an expiration condition, and a revocation path that actually works during the test. A control is not effective if the agent can keep acting after access should have ended.

Practitioner takeaway: The right test is not “can the model be tricked?”, it is “can the model be tricked into using authority it should never have had in the first place?”

Related resources from NHI Mgmt Group

• When should organisations combine AI with PAM instead of relying on manual reviews?
• Why do AI agents increase non-human identity risk in existing IAM programmes?
• Why do AI agents create more IAM risk than ordinary developer tools?
• How does the rise of AI identities impact traditional IAM systems?