Who is accountable when an AI factory identity is misused?

Why This Matters for Security Teams

AI factory identities are not abstract assets. They are the service accounts, API keys, orchestration tokens, and admin credentials that let model pipelines move data, call tools, and deploy workloads. When one is misused, the blast radius can include model operations, customer data, and downstream systems. NIST’s Cybersecurity Framework 2.0 is useful here because it frames accountability around governance and control ownership, not just authentication events.

That distinction matters because many teams assume a stronger perimeter or a valid login proves control. It does not. If the identity lifecycle is weak, the organisation still owns the failure even when an attacker appears to act “within” approved access paths. NHIMG has documented how exposed non-human credentials can be abused rapidly in real-world incidents, including the LLMjacking research and broader patterns in the 52 NHI Breaches Analysis. In practice, many security teams discover ownership gaps only after a compromised token has already been used to pivot through the AI stack, rather than through intentional identity governance.

How It Works in Practice

Accountability starts with control-plane ownership. The organisation running the AI factory is responsible for defining who can create, approve, rotate, revoke, and inspect each identity used by the agentic and model-serving environment. That includes machine identities for pipelines, service accounts for inference orchestration, and privileged access used by operators. Strong zoning and MFA reduce exposure, but they do not assign responsibility by themselves.

In mature environments, the practical answer is to treat each AI factory identity like a governed workload asset with named owners, documented purpose, and auditable boundaries. Current guidance suggests pairing lifecycle management with request-time authorization, so access is evaluated using context such as workload, task, and destination rather than just a standing role. This aligns with NIST-style governance and the broader NHI guidance in NHIMG’s Ultimate Guide to NHIs and Top 10 NHI Issues.

• Assign a business and technical owner for every AI factory identity.
• Separate creation, approval, and revocation duties so one team cannot self-authorize unchecked access.
• Log token issuance, privilege changes, and tool calls in a way that supports attribution after misuse.
• Use short-lived credentials where possible, because long-lived secrets extend the window of misuse.
• Map each identity to a specific workload boundary so misuse can be traced to a control owner.

For implementation, teams usually combine central secrets governance, workload identity, and policy enforcement at the control plane. The lesson from incidents such as the DeepSeek breach is that hidden credentials and weak boundary control turn identity misuse into a governance failure, not just an incident response problem. These controls tend to break down when AI platforms are stitched together across cloud accounts and teams because no single owner can prove who approved, issued, and monitored the credential path.

Common Variations and Edge Cases

Tighter identity control often increases operational overhead, requiring organisations to balance auditability against delivery speed. That tradeoff is especially visible in shared AI platforms, where platform engineering, security, and product teams each touch the same control plane. Best practice is evolving, but there is no universal standard for how much delegated autonomy an AI factory should have before direct human approval is required.

One common edge case is the managed service or outsourced model. Even if a vendor operates part of the stack, accountability usually remains with the organisation that chose the architecture, granted the access, and accepted the risk. Another edge case is emergency access: if operators use break-glass credentials without tight logging, attribution becomes weaker even though the event was legitimate. Security teams should also expect confusion where identities are reused across environments, because a leaked credential may look like one service but actually unlock several.

Practical accountability improves when teams document identity ownership in the same way they document data stewardship. That means every AI factory identity should have a revoke path, a review cadence, and a control owner who can answer who approved it, why it exists, and what happens when it is abused. In the field, the hardest failures are rarely caused by missing authentication alone, but by unclear ownership after the first suspicious token use.

Related resources from NHI Mgmt Group

• Who is accountable when an AI agent or workflow executes privileged actions under a forged identity?
• Who is accountable when a Vertex AI identity is over-privileged?
• Who is accountable when a secure email gateway misses an identity-led attack?
• Who is accountable for reducing password reset exposure in a healthcare identity programme?