Who should own AI red teaming when identity and security controls are involved?

Why This Matters for Security Teams

ai red teaming becomes harder, and more valuable, when the system can act with credentials, delegated access, or tool integrations. The question is not only whether the model can be tricked, but whether a prompt, workflow, or plugin path can turn into real access, data movement, or privilege escalation. That is why identity and security controls belong in the red team scope, not as an afterthought. Current guidance suggests treating these systems as operational attack surfaces, not just model evaluation targets.

NHI Management Group’s research shows why this matters in practice: Ultimate Guide to NHIs reports that 80% of identity breaches involved compromised non-human identities such as service accounts and API keys. That kind of failure cannot be assessed properly by model teams alone, because the impact often sits in access paths, secrets handling, and over-privileged integrations. Security teams are usually better positioned to test those failure modes, while product and platform owners understand the intended workflows. In practice, many teams discover this only after a model action has already reached a token, API, or admin path.

How It Works in Practice

Effective ownership is shared, but not blurred. Red teaming should have one accountable coordinator, usually security, with product, legal, identity, and platform teams contributing to scoped test plans, escalation paths, and remediation. The security function typically defines abuse cases, evidence requirements, and severity thresholds. Identity owners validate whether a finding is actually exploitable, what permissions were present, and whether the access path violates least privilege or segregation of duties.

For systems that use secrets or delegated access, the test plan should include the full chain: credential issuance, token scope, refresh behaviour, logging, revocation, and downstream tool permissions. This is where framework alignment matters. The OWASP guidance for agentic and non-human identity risk, plus control families in Anthropic Frontier Red Team — Claude Mythos technical analysis, both point toward testing the interaction between model behaviour and real access. In parallel, NIST’s AI Risk Management Framework supports governance, mapping, and measurement of AI-related risk, while NIST Zero Trust thinking reinforces verifying each request rather than trusting the surrounding environment.

Practically, the red team should ask questions such as:

• Can the agent obtain more access than its stated task requires?
• Can a malicious prompt trigger use of a privileged token, service account, or connector?
• Are logs sufficient to tell which identity performed the action and why?
• Can access be revoked quickly without breaking the whole workflow?

NHIMG’s Top 10 NHI Issues is useful here because it reinforces the operational failures that make red team findings hard to fix: weak rotation, poor visibility, and excessive privilege. These controls tend to break down when the agent is connected to third-party SaaS tools or CI/CD automation because the blast radius is spread across multiple owners and no single team sees the full access path.

Common Variations and Edge Cases

Tighter red team ownership often increases coordination overhead, requiring organisations to balance speed against accountability. That tradeoff becomes most visible when legal, procurement, and vendor management are also involved, especially for external model APIs or managed agent platforms. In those cases, the question is not just who runs the test, but who can approve disclosure, reproduce the finding, and change the integration safely.

There is no universal standard for this yet. Best practice is evolving toward a model where security owns the testing program, while identity and platform teams own the control fixes and the evidence trail. For highly regulated environments, legal may need to approve data handling during the test, and product may need to decide whether the agent’s behavior is acceptable even when technically “working as designed.”

Edge cases also show up when red teaming spans internal systems and third-party services. If an agent can chain tool calls across vendors, ownership must include the teams that manage OAuth apps, API keys, and connector scopes. That is why the State of Non-Human Identity Security matters: it highlights the visibility gap that makes identity-related findings difficult to triage. Shared ownership is the right model, but only if each team has a clear lane for decision-making and remediation.

Related resources from NHI Mgmt Group

• Who should own accountability for AI safety controls when models can call tools?
• What are the emerging security controls needed for Agentic AI identity governance?
• Why is single-provider AI agent governance not enough for enterprise security?
• How should security teams handle risks from AI browser extensions?