Who should own security quality when code generation shifts from developers to agents?

Why This Matters for Security Teams

When code generation shifts from developers to agents, the risk does not disappear, it changes shape. Security quality now depends on whether organisations can govern prompts, constrain tool use, review outputs, and detect unsafe changes before they reach production. The right question is not who typed the code, but who is accountable for its provenance, validation, and downstream impact. Current guidance in the NIST AI Risk Management Framework and related agentic AI guidance points toward shared accountability with explicit control ownership.

Teams often miss that AI-generated code can introduce security debt at machine speed. A prompt that produces a working function can still embed insecure defaults, hidden dependency risks, or logic that bypasses intended policy. Security leaders also need to account for the fact that agents may call tools, fetch context, or alter code paths in ways that are not obvious in a standard pull request review. The ownership model therefore has to cover both the quality of the output and the trustworthiness of the generation process. In practice, many security teams encounter agent-produced defects only after an incident review, rather than through intentional governance of the build workflow.

How It Works in Practice

Operational ownership usually works best as a three-layer model. Engineering owns the development workflow, secure prompt design, and review gates. Security owns policy, detection logic, and acceptance criteria for what qualifies as safe enough to merge. Platform or DevSecOps teams often act as the control plane, implementing scanning, provenance checks, and audit logging. This is consistent with the direction of the OWASP Agentic AI Top 10, which treats agent misuse, tool abuse, and output integrity as security concerns rather than purely engineering concerns.

In practice, the control set should include:

• Prompt and instruction review for unsafe patterns, hidden objectives, and unbounded autonomy.
• Code scanning for secrets, vulnerable dependencies, unsafe deserialisation, and weak access control.
• Change approval rules that require human sign-off for high-risk modules, infrastructure code, and identity logic.
• Provenance tracking so teams can trace which agent, prompt, model version, and context source produced a change.
• Detection for anomalous code patterns, repeated agent failures, and unexpected tool calls during generation.

Security quality also depends on model risk management. The MITRE ATLAS adversarial AI threat matrix helps teams think through prompt injection, model manipulation, and abuse of AI-enabled workflows, while CSA MAESTRO agentic AI threat modeling framework is useful for mapping trust boundaries around agent actions and delegated permissions. Where code generation is connected to release automation, the ownership boundary should extend into CI/CD, because the agent is effectively part of the software supply chain. These controls tend to break down when agents can modify shared infrastructure, approve their own outputs, or operate with broad repository write access because oversight becomes too coarse to catch misuse quickly.

Common Variations and Edge Cases

Tighter oversight often increases release friction, requiring organisations to balance delivery speed against the cost of deeper review. That tradeoff is real, especially when teams use agents for low-risk productivity work as well as for security-sensitive code generation. Best practice is evolving, and there is no universal standard for how much autonomy is acceptable across all repositories or environments.

Some edge cases need special handling. For example, generated test code may tolerate lower review overhead than generated authentication logic, payment flows, or access control rules. Similarly, an internal prototype may justify broader agent autonomy than production infrastructure code. Where agent output is used in regulated or high-impact environments, the accountable owner should be named explicitly and the acceptance criteria should be documented. The same applies when an agent can retrieve context from documents, tickets, or codebases that contain secrets or sensitive personal data.

Security and engineering should also avoid the false assumption that vendor tooling substitutes for governance. Tooling can reduce noise, but it cannot decide whether a code change is acceptable in context. Organisations should treat agent-produced code as untrusted until it passes the same security intent checks applied to human-written code, with added scrutiny for provenance and tool use. The emerging guidance from the OWASP Top 10 for Agentic Applications 2026 and the NIST AI Risk Management Framework supports that position, but organisations still need to decide where human approval remains mandatory.

Related resources from NHI Mgmt Group

• How should security teams inventory AI agents across SaaS, cloud, and low-code platforms?
• How should security teams govern developer agents that can act across code, build, and deployment systems?
• How should security teams govern AI agents that have their own sandboxes?
• How should security teams govern agents that can describe their own architecture?