Who is accountable when an AI system violates Separation of Duties?

Why This Matters for Security Teams

Separation of Duties is not only an access-control question. When an AI system can trigger approvals, call tools, and chain actions without the same human checkpoints used in manual workflows, the control objective shifts from “who has access” to “who approved the machine-enabled path.” That makes accountability a governance issue across business ownership, identity design, and system operation, not a single team’s failure. Current guidance from the NIST Cybersecurity Framework 2.0 still helps frame ownership, but it must be applied to autonomous workflows rather than static role maps.

That distinction matters because AI-enabled abuse often escalates faster than human review cycles can react. NHIMG has shown how exposed credentials can be weaponised within minutes in the LLMjacking threat pattern, and how hidden secrets and overbroad access compound operational risk in the DeepSeek breach. In practice, many security teams encounter SoD violations only after an autonomous workflow has already completed the prohibited action path, rather than through intentional design review.

How It Works in Practice

Accountability should be assigned to the parties that designed, approved, and operated the control path. The business owner owns the process outcome, the control owner owns the SoD rule, the system owner owns the implementation, and the identity team owns the credentials or workload identity that made the action possible. For AI systems, that typically means the review must include the agent’s tool permissions, runtime policies, escalation paths, and any human-in-the-loop checkpoints that were assumed but not enforced.

Practically, the question is whether the system was built to prevent a single identity from initiating conflicting steps, or whether it merely trusted the workflow design to stay compliant. Stronger patterns use policy-as-code, runtime authorisation, and short-lived credentials so that a system can only perform the next approved action, not hold standing authority for a full process. That approach aligns with the way NIST describes risk-based governance in the NIST Cybersecurity Framework 2.0 and with NHIMG analysis of how compromised NHIs are reused across AI and application paths in LLMjacking.

• Map each prohibited SoD combination to a specific workflow step, not just a role.
• Assign explicit control ownership for approval design, identity issuance, and runtime enforcement.
• Use just-in-time access and revocation so AI systems do not retain standing privilege across tasks.
• Log the exact action path, tool call, and decision context so accountability is auditable after the fact.

These controls tend to break down when an AI agent can invoke multiple downstream services through delegated tokens, because the original approver no longer sees the full execution chain.

Common Variations and Edge Cases

Tighter SoD enforcement often increases workflow friction, requiring organisations to balance fraud prevention and operational speed against automation latency. That tradeoff is especially visible when AI is used for finance, procurement, or privileged operations, where a single workflow may legitimately need to prepare, approve, and execute different steps without violating the spirit of the control.

Best practice is evolving for these cases. Some organisations treat the AI system as a controlled tool under the business owner’s authority; others treat it as a separate actor whose actions must be independently constrained. There is no universal standard for this yet, but current guidance suggests that if the system can materially influence both sides of a conflicting control, human approval alone is not enough. The safer approach is to separate the model’s reasoning, the action executor, and the approval authority.

That nuance matters when the system uses shared service accounts, federated tokens, or broad API scopes. In those environments, SoD can appear satisfied on paper while being violated in execution. The lesson from NHIMG research on the DeepSeek breach is that hidden exposure and weak governance often travel together, so accountability must be documented before the workflow goes live, not assigned after an incident.

Related resources from NHI Mgmt Group

• What makes agentic AI an NHI governance issue?
• Why is NHI governance critical in the age of AI attacks?
• Why is single-provider AI agent governance not enough for enterprise security?
• How should security teams handle risks from AI browser extensions?