Who is accountable for agent behaviour when coding agents can shape production code and system access?

Why This Matters for Security Teams

Accountability for coding agents is not a theoretical governance question. Once an agent can generate code, open files, call tools, or trigger deployments, it can influence production outcomes in ways that resemble privileged software rather than a passive assistant. The control problem is therefore not just about prompt quality; it is about who approved the agent’s scope, how its actions are constrained, and who can evidence those decisions.

Current guidance from the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 points toward lifecycle governance, not after-the-fact troubleshooting. That means defining the agent’s permitted tasks, constraining the identities and secrets it can touch, and mapping human approvals to the highest-risk actions. It also means deciding whether the agent is merely recommending changes or actually executing them. Those are different accountability models, and many incidents arise when organisations blur them.

In practice, many security teams encounter agent overreach only after an unsafe commit, an unintended API call, or a production change has already occurred, rather than through intentional governance.

How It Works in Practice

In operational terms, accountability should be assigned to the enterprise function that authorises the agent’s use case and risk boundary, typically with shared responsibility across engineering, security, and compliance. The engineering owner defines what the agent may do, security defines what it must never do, and compliance ensures the evidence trail is sufficient for audit and incident review. This is consistent with the control logic in NIST SP 800-53 Rev 5 Security and Privacy Controls, where access, logging, and change management are separated into reviewable control families.

A practical model usually includes:

• Named business and technical owners for the agent’s actions and outputs.
• Pre-approved tool scopes, repository boundaries, and environment restrictions.
• Separate approval for high-risk actions such as merges, deployments, secret handling, and privilege changes.
• Immutable logging for prompts, tool calls, approvals, and policy decisions.
• Periodic review of whether the agent’s actual behaviour still matches its authorised purpose.

This is where agent governance intersects with NHI security. If the agent uses tokens, keys, service accounts, or delegated permissions, those credentials should be treated as non-human identities with explicit lifecycle control, not as incidental implementation details. The OWASP Non-Human Identity Top 10 is useful here because it frames identity sprawl, secret exposure, and privilege drift as first-order risks. For threat modeling, the MITRE ATLAS adversarial AI threat matrix and the CSA MAESTRO agentic AI threat modeling framework help teams reason about manipulation of agent behaviour, tool abuse, and chained actions across systems.

These controls tend to break down when coding agents are connected directly to production repositories, deployment pipelines, and privileged cloud credentials in a flat trust environment because the agent can move from suggestion to execution without a separate authorization barrier.

Common Variations and Edge Cases

Tighter agent control often increases delivery overhead, requiring organisations to balance development speed against the need for defensible oversight. That tradeoff is real, especially where teams want autonomous code generation but also need auditability and segregation of duties.

One common edge case is the “copilot versus operator” distinction. If the system only drafts code, the human author remains the accountable decision-maker. If the agent can open pull requests, approve changes, or deploy releases, accountability shifts toward the enterprise owner that permitted that authority, even if a developer initiated the workflow. Another grey area is vendor-managed agent platforms. Best practice is evolving, but there is no universal standard for assigning final accountability when the provider hosts the model and the customer authorises operational access. The customer still owns the business risk, while the provider may retain responsibility for platform integrity and contractual controls.

Identity delegation creates another nuance. If the agent is acting through a shared service account or a long-lived token, the risk is not just misuse by the model but unclear attribution after the fact. That is why evidence of who approved the agent, what permissions were granted, and when those permissions were revoked matters as much as model output quality. For teams formalising policy, NIST AI Risk Management Framework remains the most useful starting point because it supports governance, mapping, measurement, and management across the full lifecycle.

Where systems mix autonomous code generation, CI/CD access, and privileged secrets, accountability becomes easiest to lose precisely because the agent looks like a productivity tool rather than a production actor.

Related resources from NHI Mgmt Group

• What do teams get wrong about AI coding agents generating access-related code?
• Who is accountable when an AI agent in a pipeline leaks credentials and enables code push access?
• How should security teams govern coding agents that already have access to production tools?
• Who is accountable when an AI coding tool introduces insecure code into production?