Who is accountable when a third-party AI agent misbehaves in production?

The organisation using the agent remains accountable for the outcomes, even if a vendor supplies the platform. Security, legal, compliance, and business owners should share responsibility for controls, monitoring, and incident response. If the agent can affect customers or regulated data, accountability cannot be outsourced with the technology.

Why This Matters for Security Teams

When a third-party AI agent acts on live systems, the risk is not limited to model quality. It can touch customer data, approvals, payments, tickets, code changes, or privileged workflows. That makes accountability a governance issue, not just a procurement issue. Current guidance from the NIST AI Risk Management Framework is clear that organisations deploying AI must define, monitor, and manage risk across the full lifecycle, including deployment and operations.

The common mistake is assuming the vendor owns the failure mode because the vendor built the system. In practice, the enterprise controls the data, the permissions, the business context, and often the decision to let the agent act. If those elements are poorly governed, blame may be shared contractually, but operational responsibility remains with the organisation that put the agent into production. That distinction matters when incident response, customer communication, or regulatory reporting begins.

Security teams also need to distinguish between a model error and an access failure. A hallucinated answer is bad; an agent with excessive privileges is worse. The first is a quality issue, the second is an exposure issue. In practice, many security teams encounter agent accountability only after a misrouted action, unsafe approval, or data leakage has already occurred, rather than through intentional pre-production governance.

How It Works in Practice

Accountability for a third-party AI agent should be defined before the agent is allowed to act. That means assigning owners for the use case, the controls, the logs, the vendor relationship, and the incident response path. The organisation should not rely on the vendor’s general assurances. Instead, it should establish measurable controls that map to the agent’s actual permissions and failure modes.

Operationally, this usually means treating the agent like a privileged service with explicit guardrails. The most effective programmes define what the agent may read, write, approve, or trigger, then verify that those boundaries are enforced at runtime. Where the agent uses secrets, tokens, or service accounts, those credentials need the same discipline applied to other non-human identities. This is especially important because agent behaviour often depends on delegated access rather than direct human supervision.

• Assign a business owner, security owner, and technical owner for each production agent.
• Document the exact actions the agent is authorised to perform and where human approval is required.
• Monitor prompts, tool calls, outputs, and downstream actions for drift or abuse.
• Retain logs that support audit, legal review, and incident reconstruction.
• Test rollback, kill switch, and containment procedures before go-live.

Threat modelling should also reflect agent-specific attack paths. The OWASP Agentic AI Top 10 and the MITRE ATLAS adversarial AI threat matrix are useful for identifying prompt injection, tool abuse, and unsafe autonomy scenarios. These controls tend to break down when a third-party agent is connected to high-trust systems through broad API scopes because the enterprise cannot easily detect whether the failure began in the model, the integration, or the delegated credential.

Common Variations and Edge Cases

Tighter control often increases operational overhead, requiring organisations to balance autonomy against auditability. That tradeoff becomes sharper when the agent is customer-facing, interacts with regulated data, or can trigger financial or production-impacting actions. In those cases, best practice is evolving toward shared governance, where legal, risk, security, and product teams all sign off on the operating model rather than leaving it to engineering alone.

There is no universal standard for this yet, but the practical pattern is consistent: the more authority the agent has, the stronger the accountability structure must be. For low-risk tasks such as summarisation or internal drafting, lighter oversight may be acceptable. For agents that can modify records, send instructions, or execute transactions, the organisation should require stronger approval gates, evidence retention, and post-action review. The CSA MAESTRO agentic AI threat modeling framework is useful here because it frames autonomy as a risk surface, not just a feature.

Contract terms can clarify vendor obligations, but they do not remove accountability for the deployed outcome. The real edge case is when a vendor-hosted agent operates under enterprise credentials or processes enterprise data. In that situation, the organisation is still accountable for choosing the use case, approving the access, and monitoring the behaviour. For controls design, the relevant question is not who wrote the model, but who allowed it to act. That is where the NIST control environment and NIST SP 800-53 Rev 5 Security and Privacy Controls remain highly practical.

Related resources from NHI Mgmt Group

• Who is accountable when an AI agent acts on instructions from a third-party skill update?
• Who is accountable when an AI agent deletes production data?
• Who is accountable when sensitive data is retained in a third-party AI tool?
• Who should be accountable when a third-party identity chain exposes production credentials?