AI-generated code is reshaping AppSec and software trust models

At a glance

What this is: This is an analysis of how AI is changing application development and application security, with the key finding that AI is becoming part of both the developer workflow and the software stack itself.

Why it matters: It matters because IAM, NHI, and AppSec teams now have to govern not just human developers, but AI agents, models, and tool integrations that can create, access, and modify code and data.

By the numbers:

• 81% of developers surveyed reported using AI-powered coding assistants.
• 96% of security and software development professionals report that their companies use GenAI-based solutions for building or delivering applications.
• 40% of new enterprise production software will be created with vibe coding techniques and tools by 2028.

👉 Read LEGIT Security's analysis of how AI is changing software development and AppSec

Context

AI-assisted development changes the security problem before code ever reaches production. When code completion, prompt-driven generation, and agentic workflows become part of delivery, the question shifts from whether developers can write faster to whether organisations can still control what gets created, reviewed, and deployed. That creates a governance issue for application security, but it also intersects with identity governance when AI systems are allowed to act as software participants rather than passive tools.

The article argues that AI is becoming part of the application architecture itself, not just a productivity layer around it. That means models, LLMs, and agents may fetch data, call tools, and interact with other systems as part of normal software behaviour, which makes workload identity, access boundaries, and secret handling more important, not less.

For most enterprises, the starting point described here is becoming typical rather than exceptional: AI is no longer an experimental edge case in development pipelines, it is entering mainstream software creation and runtime interaction.

Key questions

Q: How should security teams govern AI-generated code in production pipelines?

A: Security teams should treat AI-generated code as a controlled identity event, not just a development artifact. Require human approval, traceable authorship, scoped workload identities, and evidence of intent before production promotion. The goal is to preserve provenance and limit blast radius when generated logic behaves unexpectedly.

Q: Why do AI agents complicate IAM and data security controls?

A: Because the core controls were built for human sessions and file-centric data movement, while agents act continuously, inherit permissions, and reason over data in context. That breaks the assumptions behind IAM, ITDR, DSPM, and DLP. The practical result is false confidence unless teams govern permissions, context, and action paths together.

Q: What breaks when AI coding assistants work in a poorly structured codebase?

A: The assistant loses context, spreads change across the wrong files, and makes it harder for reviewers to understand impact. Poor structure increases defect rates, hides security-sensitive changes, and raises the chance that a seemingly small task will alter a large amount of code. Clean structure is a control, not just a preference.

Q: Should organisations use the same access model for humans and AI agents?

A: No. Human access models are built around stable roles and review cycles, while AI agents often need contextual, task-specific permissions that change quickly. Treating them the same usually leads to over-permissioning or constant exceptions. Organisations should separate identity proof from authorization design and apply resource-level controls for agents.

Technical breakdown

AI code generation and the changing trust boundary

AI coding assistants, prompt-based generation, and agent-supported workflows compress the distance between intent and production code. The security issue is not simply that AI can write code faster. It is that more code is created through probabilistic generation, with less human inspection of intermediate decisions and more dependence on the surrounding guardrails. In AppSec terms, the trust boundary moves from the individual developer to the entire development system, including prompts, IDE integrations, source control, CI/CD, and policy checks. That makes provenance, review depth, and change accountability harder to preserve.

Practical implication: organisations need policy and review controls that inspect AI-generated changes before merge, not after release.

Vibe coding and the erosion of explicit design controls

Vibe coding shifts development from line-by-line construction to instruction-driven assembly, often with natural language prompts producing functions, tests, or full features. That changes the control model because the human is no longer the sole author of implementation detail. The risk is not only poor code quality. It is also hidden dependencies, weak security assumptions, and unclear ownership of generated logic. For security teams, this resembles a governance problem more than a tooling problem because the artefact being reviewed is the output of a chain of prompts, models, and execution context, not just source code.

Practical implication: teams should record prompt-to-code lineage for high-risk changes so they can review how sensitive logic was produced.

AI agents as software actors in the application stack

When agents can query data, call tools, or coordinate with other agents, they behave less like static features and more like runtime participants. That introduces an identity and authorisation question: what is allowed to act, on whose behalf, with which credentials, and for how long? This is where AppSec overlaps with IAM and NHI governance. AI systems that can initiate actions need scoped identities, auditable entitlements, and clear boundaries between human approval and machine execution. Without that, the software stack inherits a new class of unmanaged non-human behaviour.

Practical implication: treat AI agents as governed identities with explicit permissions, logging, and revocation paths.

Threat narrative

Attacker objective: The attacker or failure mode seeks to turn accelerated AI-assisted development into a larger attack surface, causing insecure code, excessive access, or unauthorised system behaviour.

1. Entry begins when AI-generated code, agent output, or prompt-driven changes enter the delivery pipeline with insufficient review or policy enforcement.
2. Escalation occurs when generated code or connected agents inherit broad runtime access to data, APIs, or development secrets that were never scoped for machine use.
3. Impact follows when the application exposes data, performs unauthorised actions, or propagates insecure logic into production at scale.

NHI Mgmt Group analysis

AI-assisted development creates governance debt faster than it creates code value. The article is right to frame AI as a productivity layer, but the security consequence is accumulation of hidden decision points across prompts, models, and agents. That means classic review gates see only the final artefact, not the generation chain that produced it. The practical conclusion is that AppSec programmes need provenance and policy enforcement that operate before code becomes an approved change.

AI-generated software is also an identity problem because machine actors are now inside the application path. Once agents can query systems, write code, or trigger tasks, they need identity, authorisation, and auditability like any other non-human identity. The distinction between a developer tool and a runtime actor is no longer stable enough for control design. Practitioners should treat these systems as governed identities with least privilege and revocation paths.

Application security now has to manage the trustworthiness of outputs, not just the correctness of code. AI changes the assurance model because risky behaviour can be introduced through generated tests, scaffolding, dependencies, or orchestration logic that looks structurally valid. This creates a named concept worth tracking: AI delivery trust gap: the space between accepted code and understood provenance, where security review no longer sees how the logic was produced. Teams should close that gap with traceability and policy checks.

Security teams should stop asking whether AI will increase development velocity and start asking where the new control boundary sits. If the boundary remains at the developer workstation, then AI systems effectively sit outside governance. If it moves into the pipeline, then code review, entitlement management, and secret handling become part of the same control plane. The right conclusion is to redesign governance around AI-mediated delivery rather than bolting controls onto it later.

What this signals

AI delivery trust gap: teams should expect AI-assisted development to create more approved-looking changes that are less understood at review time. That shifts the programme focus toward lineage, policy, and entitlement control across the pipeline, not just code quality.

If AI agents are allowed to query systems or execute tasks, they should be governed as non-human identities with explicit permission scopes and revocation paths. The control challenge is no longer only detecting risky code, but also constraining the machine actors producing and using it.

The practical signal for security leaders is that AppSec, IAM, and secrets governance are converging around the same delivery pipeline. Organisations that keep those controls separate will miss the places where AI turns development speed into security debt.

For practitioners

• Enforce provenance checks on AI-generated changes Require prompts, model interactions, and generated artifacts to be traceable for high-risk code paths so reviewers can see how the change was produced.
• Scope non-human access for AI agents Assign explicit permissions to AI agents that query data or call tools, then review those permissions as if the agent were a workload identity.
• Add security gates before merge, not just before release Use policy checks, secret scanning, and dependency review at the point where AI-generated code enters source control, because post-build inspection is too late for many failure modes.
• Map which development tasks can be delegated to agents Define where agentic workflows may generate code, run tests, or change configuration, and block autonomous execution for sensitive assets or privileged environments.

Key takeaways

• AI changes AppSec by moving trust boundaries into the generation process, where code is produced by prompts, models, and agents rather than only by human authors.
• The scale of adoption is already high, which means governance gaps in provenance, access, and review will affect mainstream delivery rather than isolated experiments.
• Security teams need to govern AI-generated code and AI actors together, because the same pipeline now creates both software and the identities that act on it.

Key terms

• AI Delivery Trust Gap: The gap between code that appears valid in review and code whose origin, prompts, model inputs, and generation path are not fully understood. It matters because modern delivery pipelines may accept artefacts without enough provenance to explain how sensitive logic was created.
• Agentic workflow: An agentic workflow is a sequence of tasks executed by an AI agent with some level of tool access and decision authority. In security terms, the workflow matters because it can span multiple systems, identities, and permissions, which makes attribution and revocation harder than with ordinary automation.
• AI-Generated Code Provenance: AI-generated code provenance is the ability to trace where generated code came from, how it was transformed, and what validation it passed before release. It matters because teams need evidence that machine-assisted code met the same policy expectations as hand-written code.

What's in the full article

LEGIT Security's full article covers the operational detail this post intentionally leaves for the source:

• Examples of how AI coding assistants change developer workflow and code review expectations
• The reported survey breakdowns on GenAI adoption in application delivery and developer usage
• Gartner's 2028 vibe coding forecast and how the vendor frames its relevance to AppSec
• The article's own view of AI as both a security challenge and a path to more efficient application protection

👉 LEGIT Security's full article expands on AI coding assistants, vibe coding, and agentic development workflows.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, IAM, and secrets management. It is designed for practitioners who need to align identity controls with modern development and runtime risk.