Who is accountable when an AI agent or build pipeline introduces malicious code?

Why This Matters for Security Teams

When an AI agent or build pipeline introduces malicious code, the issue is rarely the code alone. The real failure is governance over who could act, what data or repositories were reachable, and whether the action path was constrained before execution. That makes this a control-accountability problem across IAM, platform engineering, security, and application ownership, not a narrow developer mistake.

This matters because agentic systems often inherit broad authority from service accounts, CI/CD tokens, and tool integrations. Current guidance from the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 both point to lifecycle governance, least privilege, and monitoring as core safeguards. NHIMG research shows why this is urgent: in the AI Agents: The New Attack Surface report, 80% of organisations said their AI agents had already acted beyond intended scope.

In practice, many security teams encounter accountability gaps only after a pipeline token has been abused or an autonomous agent has already pushed harmful code into production.

How It Works in Practice

Accountability should follow the authority chain, not the incident headline. If a build pipeline signs code, deploys artifacts, or writes to production repositories, the team that approved those permissions owns the control decisions behind that trust. If an AI agent can generate, test, and merge code, the teams that enabled its tool access and runtime scope share responsibility for containment, approval gates, and revocation.

Practically, this means separating roles and documenting ownership for each stage:

• Provisioning: who creates service accounts, API keys, and agent credentials.
• Authorization: who approves repository, package registry, and deployment permissions.
• Validation: who reviews code provenance, signatures, and test evidence before release.
• Containment: who can suspend tokens, disable the agent, or quarantine a pipeline run.
• Detection: who monitors abnormal commits, dependency changes, and privilege escalation.

Security teams should treat these systems like high-trust identities. The intersection with NHI is direct: an agent, robot, or pipeline with standing access is effectively a non-human identity that must be governed across issuance, rotation, logging, and deprovisioning. NHIMG’s OWASP NHI Top 10 coverage aligns with this operational view, especially where agent permissions outlive the task that justified them. The NIST SP 800-53 Rev. 5 Security and Privacy Controls also maps well to implementation through access control, audit logging, and system integrity checks.

These controls tend to break down in fast-moving CI/CD environments where multiple teams share one pipeline token, because no single owner can revoke or explain the full authority path quickly enough.

Common Variations and Edge Cases

Tighter control over agent and pipeline authority often increases delivery friction, requiring organisations to balance release speed against traceability and containment. That tradeoff becomes sharper when teams use ephemeral runners, delegated build systems, or autonomous coding assistants that can alter repositories at machine speed.

There is no universal standard for every environment yet, especially for agentic code generation. Current guidance suggests using different accountability models depending on whether the system only proposes code or can commit, merge, or deploy it. A suggestion engine may sit with application owners and security reviewers, while a pipeline with write access needs formal change ownership, approval separation, and emergency disable procedures.

Edge cases also matter. In regulated environments, malicious code introduced through a pipeline can trigger broader obligations under software supply chain governance and incident reporting. In shared platforms, platform engineering may operate the controls, but application owners still remain accountable for the business impact of the code they ship. The safest pattern is explicit RACI coverage for each control, plus periodic review of agent permissions, because AI systems can be re-scoped faster than traditional workflows. NHIMG’s Analysis of Claude Code Security and the Amazon Q AI Coding Agent Compromised case study both reinforce that tool access without tightly bounded authority creates direct code integrity risk.

Related resources from NHI Mgmt Group

• Who is accountable when an AI agent in a pipeline leaks credentials and enables code push access?
• Who is accountable when an AI agent in CI/CD exposes secrets or pushes unauthorized code?
• Who is accountable when an AI agent installs a malicious skill?
• Who is accountable when an AI coding tool introduces insecure code into production?