Who is accountable when an AI agent exfiltrates data after being manipulated by attacker content?

Why This Matters for Security Teams

When an AI agent exfiltrates data after attacker content manipulates its instructions, the core issue is not just malicious input. It is that the agent had enough trust, tool access, and data reach to turn a prompt into a security event. Static attribution often fails because the agent may act under a legitimate user session while crossing boundaries the user never intended. That is why current guidance from the OWASP Top 10 for Agentic Applications 2026 and the OWASP Agentic Applications Top 10 treats prompt injection, tool misuse, and overbroad authority as first-order risks, not edge cases.

For accountability, the important question is which control failed: privilege design, tool governance, instruction-channel filtering, or data egress constraints. The answer usually spans the team that approved the agent, the owners of the connected systems, and the programme that allowed the agent to inherit broad permissions without task-level limits. In practice, many security teams discover the accountability gap only after the exfiltration has already been investigated as a “user action.”

How It Works in Practice

Operational accountability for agentic systems starts by treating the agent as a workload with its own identity, not as a human proxy. The agent should authenticate through workload identity and receive task-scoped privileges at runtime, ideally through short-lived credentials rather than standing access. That aligns with the direction of the NIST AI Risk Management Framework and the CSA MAESTRO agentic AI threat modeling framework, both of which emphasise governable risk, traceable behaviour, and bounded autonomy.

In practice, that means three things:

• Use workload identity for the agent, such as SPIFFE-style identities or scoped OIDC tokens, so logs show what the agent is rather than who happened to launch it.
• Issue just-in-time credentials per task and revoke them automatically when the task ends or the policy context changes.
• Evaluate authorisation at request time with policy-as-code so the agent’s next action is checked against current context, not a preapproved blanket role.

That design makes accountability measurable. Security teams can map each tool call, token issuance, and data access event to a control owner and a policy decision. The presence of attacker content then becomes evidence of an input manipulation attempt, not proof that the agent should have been allowed to export data. NHIMG’s 52 NHI Breaches Analysis shows how often identity weaknesses, not just malware, sit at the centre of breach paths, and OWASP NHI Top 10 reinforces that agent privilege boundaries must be explicit, ephemeral, and auditable.

These controls tend to break down in environments where agents can chain tools across multiple SaaS platforms, because downstream services often inherit trust from the original session without rechecking intent.

Common Variations and Edge Cases

Tighter agent controls often increase operational overhead, requiring organisations to balance autonomy against review burden and latency. That tradeoff is real, especially when teams want fast agent execution but also need provable accountability after a security event.

Current guidance suggests there is no universal standard for responsibility assignment yet, so organisations should define it contractually and technically. If an agent is deployed by one team, configured by another, and connected to a high-value data store owned by a third, accountability should be split across approval, integration, and oversight functions. The user whose prompt was manipulated is rarely the right sole owner of the incident.

Edge cases matter most when the agent acts under delegated human authority, when multiple agents collaborate, or when logs are incomplete. In those cases, the cleanest attribution model is to assign responsibility based on control failure: who granted access, who approved the data path, and who failed to constrain the instruction channel. For threat modelling, MITRE ATLAS adversarial AI threat matrix helps teams catalogue attacker techniques, while CISA cyber threat advisories provide current attacker tradecraft context for response planning.

NHIMG’s AI Agents: The New Attack Surface report is especially relevant here: it shows that many organisations still lack visibility into what agents access, which makes post-incident accountability harder than prevention. In mature programmes, the incident record should show which policy allowed the action, which secret enabled it, and which owner approved that level of autonomy.

Related resources from NHI Mgmt Group

• Who is accountable when an AI agent uses stolen signing material to access production systems?
• What is the difference between human identity governance and AI agent governance?
• When does AI agent access create more risk than it reduces?
• What is the difference between governing human access and governing AI agent access?