Who is accountable when an autonomous coding agent ships flawed code?

Accountability sits with the team that granted the agent its access and operating scope. If traces, attribution, and evaluation were not in place, the failure is a governance gap, not just a model mistake. Organisations should tie agent permissions to documented oversight controls and require evidence before expanding autonomy.

Why This Matters for Security Teams

autonomous coding agent change the accountability model because they can create, modify, and commit code with a degree of speed that outpaces manual review. That means the question is not only whether the code is flawed, but whether the organisation defined who approved the agent’s scope, who monitored its actions, and who could stop it. The NIST AI Risk Management Framework is useful here because it frames AI governance as a lifecycle responsibility, not a one-time sign-off.

Practitioners often focus on whether the agent used a secure model or a vetted prompt, but that is only part of the control picture. Accountability depends on evidence: approved use cases, traceable actions, human review thresholds, rollback procedures, and clear ownership for software changes introduced by the agent. If those controls are absent, the organisation has delegated execution without delegating governance, which creates a gap between technical capability and operational responsibility.

In practice, many security teams encounter agent-caused defects only after a production issue, not through intentional oversight of the agent’s decision path.

How It Works in Practice

Accountability should be assigned at the control layer, not to the model itself. The team that authorises the agent’s access determines its scope, and the team that operates the software delivery process must ensure review, logging, and exception handling. Current guidance suggests treating coding agents as high-risk automation that requires bounded permissions, change traceability, and explicit approval gates before code reaches protected environments. The OWASP Agentic AI Top 10 is especially relevant because it highlights failure modes such as unsafe tool use, insufficient oversight, and weak action boundaries.

A practical operating model usually includes:

• Defined business owner and technical owner for the agent’s use case.
• Role-based access to repositories, secrets, CI/CD systems, and deployment tools.
• Human approval for merges, releases, and any action that changes production state.
• Audit logs that capture prompts, tool calls, code diffs, and approvals.
• Evaluation tests that check for regressions, insecure patterns, and policy violations before autonomy expands.

This is also where software supply chain controls matter. If an agent can pull dependencies, alter build steps, or generate infrastructure code, then the organisation must validate provenance and review the resulting artefacts with the same seriousness applied to human-authored changes. The NIST SP 800-53 Rev 5 Security and Privacy Controls provides a good anchor for access control, auditing, and change management expectations, even though it was not written specifically for agents.

These controls tend to break down in fast-moving product teams that allow the agent to commit directly to main branches and deploy into environments where human review is optional.

Common Variations and Edge Cases

Tighter approval gates often increase delivery friction, requiring organisations to balance speed against the risk of unreviewed machine-generated changes. That tradeoff becomes sharper when the agent is used for low-risk refactoring versus security-sensitive work such as auth flows, secrets handling, or infrastructure-as-code. Best practice is evolving, and there is no universal standard for autonomy thresholds yet, so maturity should be increased gradually and based on evidence from evaluations, not enthusiasm for automation.

Some teams assume that if the model generated the defect, the model is at fault. That is rarely a useful accountability conclusion. The more important question is whether the organisation set policy for when the agent can act alone, when a developer must review its output, and when the system should halt and escalate. The CSA MAESTRO agentic AI threat modeling framework helps structure those decisions by tying autonomy to threat paths, controls, and operational boundaries.

Where this guidance becomes harder to apply is in multi-agent pipelines, experimental developer tooling, and environments that mix code generation with privileged secrets or deployment credentials. In those settings, accountability must be shared across platform, application, and security owners, because a single broken control can allow flawed code to move from suggestion to release without a clearly documented human decision.

Related resources from NHI Mgmt Group

• Who is accountable when an MCP-connected coding agent changes code or data outside policy?
• Who is accountable when an autonomous agent misuses access or exposes data?
• Who is accountable when an autonomous AI agent causes a security incident?
• Who should be accountable for autonomous agent activity in the enterprise?