SSDF must evolve for AI-generated code and active attack surfaces

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Sonar's blog post: SSDF assumptions no longer fit AI-generated code, because probabilistic output, review overload, and agent exposure create a need for deterministic verification before release.

Why it matters: IAM, NHI, and software security teams need to treat AI coding systems as governed producers of change, not just productivity tools, because their outputs and permissions expand the supply chain risk surface.

By the numbers:

• Sonar analyzes more than 750 billion lines of code every day across 7 million software developers.
• Sonar's research found that coding models, left unverified, produce roughly 1,200 security issues per million lines of code analyzed.
• Sonar reports that AI-generated PRs modify an average of 14x more code than traditional tools, while agentic workflows correlate with a 3-5x increase in lines added per project.

Context

Software development frameworks assume that tools behave predictably, reviewers can keep pace with human-sized changes, and control gates can reliably inspect what is being released. AI-generated code disrupts all three assumptions because it can be probabilistic, voluminous, and itself exposed to adversarial manipulation.

The identity angle matters because AI coding systems increasingly act like non-human producers inside the software supply chain, using credentials, repositories, CI/CD systems, and deployment pipelines. That turns code generation into a governance problem as much as a software quality problem.

Sonar's article argues that SSDF needs to move from trusting the generator to verifying the outcome, which is a typical response when automation starts to outpace human review capacity.

Key questions

Q: What breaks when AI-generated code is treated like normal developer output?

A: Teams can miss insecure patterns, risky dependencies, and hidden assumptions because the review process assumes human authorship. That creates a governance gap where plausible-looking code moves forward with too little scrutiny. The fix is to treat generated code as untrusted until it passes automated testing, dependency review, and explicit human approval.

Q: Why do AI coding agents increase supply-chain risk?

A: AI coding agents can choose tooling, install packages, and edit lock files in ways that may bypass the controls humans expect in a managed environment. That widens the blind spot around untracked software and makes package provenance harder to enforce. The practical response is to restrict agent permissions, inventory their workspaces, and review their install behaviour like any other privileged automation.

Q: What are the warning signs that AI code review is failing?

A: Look for large pull requests approved with minimal discussion, reviewers unable to explain the change, repeated exceptions to standards, and security checks treated as ceremonial. Those signals show that acceptance is drifting from evidence-based review toward confidence in the model’s output.

Q: How should teams govern AI systems that act like privileged insiders?

A: Treat AI systems as governed actors with explicit access scope, strong logging, and containment boundaries. If an AI can influence data or actions at speed, traditional review cycles are too slow to provide meaningful oversight. The control question becomes what it can access and what it can do.

Technical breakdown

Why probabilistic code generation breaks deterministic verification

The SSDF was written for a world where the same input should produce the same output, or at least a repeatable verification result. AI code generation is probabilistic, so a safe answer yesterday can become a risky answer today even with similar prompts. That means the generator cannot self-certify its own output, because the verification problem is no longer equivalent to the generation problem. In practice, this shifts assurance to deterministic checks that are independent of the model's internal reasoning. AI-assisted review can help triage, but it cannot be the final authority on correctness or security.

Practical implication: Treat deterministic analysis as the release gate for AI-authored code, not the model's own approval.

Why agentic code volume overwhelms human review and review workflows

Agentic workflows do not just change how code is written, they change the scale of change. Larger pull requests, more lines added per project, and higher dependency churn all raise the likelihood that review becomes superficial or inconsistent. The review process itself is the bottleneck, not the number of engineers. Once a change set exceeds what humans can examine carefully, organizations need layered verification that combines code review, software composition analysis, and CI validation. The control objective becomes coverage across failure modes, not a single perfect reviewer.

Practical implication: Use automated verification layers to make code review tractable when AI produces changes at machine speed.

How active attack surfaces emerge in AI coding pipelines

AI is not only a source of untrusted code, it is an attacker-reachable system. When agents can read repositories, tickets, build logs, or API responses, prompt injection and hidden instructions can steer the agent into unsafe actions. If those agents also hold credentials or persistent memory, an attacker can try to influence future behavior, not just one output. That turns the coding pipeline into an active attack surface rather than a passive automation layer. The security model must therefore cover access scope, instruction integrity, and the trustworthiness of the data the agent consumes.

Practical implication: Constrain agent access and validate inputs so coding assistants cannot be steered into attacker-controlled actions.

Threat narrative

Attacker objective: The attacker aims to influence AI-assisted development so that unsafe code, malicious dependencies, or pipeline actions are shipped as trusted output.

1. Entry occurs when malicious instructions are embedded in files, tickets, or API responses that an AI coding agent consumes during development workflows.
2. Credentialed access to repositories, CI/CD systems, or deployment pipelines lets the agent act on those instructions with inherited privileges.
3. The result is code modification, dependency churn, or unsafe pipeline actions that can persist into released software and broaden downstream exposure.

Breaches seen in the wild

• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.
• GemStuffer RubyGems campaign 2026: AI agents attributed to OpenAI mass-created RubyGems accounts, ran code on RubyDoc build servers and probed a flaw to steal API keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Outcome verification is now the governing principle for AI-assisted software delivery. The SSDF cannot depend on a model to validate its own work because probabilistic generation breaks the premise of deterministic assurance. That means the control conversation shifts from trusting authorship to proving security and quality before release. Practitioners should treat this as a structural change in software governance, not a tooling preference.

AI code generation creates governance debt in the software supply chain. Larger patches, faster dependency churn, and more automated pull requests mean traditional peer review is increasingly a coordination layer, not a control layer. This is where NIST CSF, NIST 800-53, and CIS Controls still matter, but they must be applied through deterministic automation rather than human inspection alone. The practitioner takeaway is to make verification scale with generation.

AI coding systems are non-human identities with privileged workflow access. Once they can read repositories, trigger builds, or modify deployment artifacts, their permissions become part of the identity governance problem. That makes this topic directly relevant to NHI oversight: the agent or assistant is not just a tool, it is a governed actor inside the delivery chain. Security teams should manage it with the same rigor they apply to other high-trust machine identities.

Active attack surfaces change the SSDF from a development standard into an adversary-aware control model. Prompt injection, memory-driven privilege escalation, and adversarial inputs are not edge cases when AI is embedded in delivery workflows. They show that secure development now depends on instruction integrity, access containment, and outcome validation together. Practitioners should assume the build pipeline can be manipulated through the model, not just through code.

From our research library:

• 92% of organisations expose NHIs to third parties, raising concerns about supply chain security, according to the Ultimate Guide to NHIs.
• Breaches involving third parties rose to 48% of all breaches, a 60% increase on the previous year, according to Verizon's 2026 Data Breach Investigations Report.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

AI-generated code has become an identity governance problem as well as a software quality problem. Once coding agents can touch repositories, pipelines, and secrets, they should be governed like other privileged non-human actors rather than treated as neutral productivity tools. That shift matters because access scope, approval boundaries, and auditability now determine whether automation creates a control gain or a control gap.

Outcome verification is the named control gap that matters here. The failure mode is not simply that AI writes imperfect code, but that the industry is still tempted to trust generation, review, and testing as if they were interchangeable with deterministic assurance. Security teams should respond by making release decisions depend on independent verification, not model confidence.

AI-assisted development also widens supply chain exposure because more code and more dependencies move through the pipeline faster than human review can scale, which makes the controls around code provenance and change approval more important than ever.

For practitioners

• Mandate deterministic post-generation verification Require every AI-authored change to pass independent static analysis, SCA, and CI checks before merge or release, regardless of who or what produced it.
• Limit repository and pipeline privileges for coding agents Scope AI coding assistants to the smallest set of repositories, build actions, and secrets needed for their task, and separate generation from deployment authority.
• Validate prompts and inputs as untrusted content Treat tickets, file content, retrieved context, and API responses as potentially malicious instructions that can steer agent behaviour.
• Track AI-authored changes as governed NHI activity Log which agent or model touched which repository, what permissions it used, and what controls approved the resulting change so reviews are auditable.

Key takeaways

• AI-generated code breaks the SSDF assumption that tools are deterministic and reviewable at human scale.
• The practical risk is not just lower code quality, but a wider attack surface created by privileged, non-human development actors.
• Security teams should make release approval depend on independent verification, controlled privileges, and auditable change paths.

Key terms

• Deterministic Verification: A verification method that produces the same enforced result every time when the required proof is present, instead of relying on human judgement or probabilistic signals. It is useful for high-blast-radius access changes because it removes discretion from the decision point.
• Agentic Code Volume: The amount of code, dependency churn, and workflow activity produced by AI agents during software development. It matters because high-volume automated change can outpace human review, pushing security teams toward layered controls that validate code before it reaches production.
• Dynamic Attack Surface: The set of externally reachable systems, services, and exposures that changes as infrastructure changes. Modern environments use load balancers, virtual hosting, and frequent deployment cycles, so the attack surface cannot be treated as fixed. Discovery and testing methods must continuously adapt to remain accurate.
• Outcome Verification: Outcome verification is the practice of checking whether a security action produced the intended state change in the source system. It goes beyond logging that an action was requested or accepted and asks whether the environment actually moved into the safer condition.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management in practical terms. It helps practitioners connect identity controls to the workflows that modern development and AI systems depend on.