Vibe coding leaves AI-generated code secure by exception

At a glance

What this is: This analysis argues that vibe coding can produce working code that still fails security tests, especially in real repositories with multi-file context and subtle historical vulnerabilities.

Why it matters: It matters because IAM, PAM, and broader security teams increasingly depend on AI-assisted development pipelines that can touch secrets, credentials, and access logic without adequate guardrails.

By the numbers:

• About 75% of developers already vibe code, according to the article's cited survey.
• The best pairing in the study passed functional tests on 61% of tasks but both functional and security tests on only 10.5%.
• The benchmark evaluated 200 real feature-request tasks across 108 GitHub projects.

👉 Read Symbiotic Security's analysis of why vibe coding is not secured by default

Context

Vibe coding is now a security governance issue, not just a developer convenience. When AI agents implement features directly in real repositories, the core question becomes whether the output is secure enough to be trusted in production, especially where code can affect authentication, secrets, and access control.

The study described in the source article matters because it moves beyond toy prompts and into repository-level work where vulnerabilities are subtle, stateful, and easy to miss. For identity teams, the important intersection is that AI-assisted coding increasingly touches IAM logic, secret handling, and policy enforcement paths that govern both human and non-human access.

Key questions

Q: How should teams govern AI agents that refactor production systems?

A: Teams should govern agentic refactoring as a constrained execution problem, not a free-form coding problem. The agent should work from explicit manifests, bounded file access, and clear stop conditions. If the change depends on sequencing, live state, or irreversible transitions, human review must control the next step before the agent proceeds.

Q: Why do functionally correct AI patches still create security risk?

A: Because a patch can satisfy tests while still violating the security invariant the original bug depended on. AI agents are often good at completing the requested feature but weak at preserving deeper constraints such as safe input handling, constant-time logic, or session boundaries. That is why security validation must be separate from feature validation.

Q: What do security teams get wrong about AI guardrails and prompt refusal?

A: They treat refusal as if it were a security boundary. It is not. A determined attacker can rephrase requests, split work into harmless fragments, or switch models until the operation succeeds. The real defence is to limit the identities, entitlements, and session scope that make the AI useful to the attacker.

Q: What should teams do before trusting AI output on code findings?

A: Teams should narrow the problem with reachability analysis, taint flow checks, and context about how the application is actually used. That reduces hallucination-like overreach and keeps the model focused on evidence rather than guesses. The best AI results come from bounded inputs, not from scanning the entire codebase blindly.

Technical breakdown

Why real repositories expose AI coding blind spots

Toy benchmarks do not reflect how software is built in production. In a real repository, an AI agent must preserve surrounding invariants, follow repository conventions, and avoid introducing regressions across many files and tests. That is where models often fail, because they optimise for local correctness rather than security properties such as constant-time checks, safe input handling, and session boundaries. The result is code that appears functional while violating the deeper assumptions that keep applications safe. Practical implication: evaluate AI-generated changes in full repository context, not as isolated snippets.

Practical implication: evaluate AI-generated changes in full repository context, not as isolated snippets.

Why passing tests does not prove the patch is secure

Functional tests answer whether the feature works, not whether it resists abuse. The study shows that AI agents can satisfy visible acceptance criteria while still leaving known weaknesses in place, including header injection, session persistence flaws, and unsafe link handling. This happens because the model can optimise for the immediate test surface and miss the control the bug previously violated. In security terms, the patch may look complete while the exploit path remains open. Practical implication: add security-specific assertions and independent analysis to the release gate.

Practical implication: add security-specific assertions and independent analysis to the release gate.

How agent guardrails differ from prompt-based security

Prompting a model to 'be secure' is not a control framework. The article shows that security reminders and CWE prompting can improve some outputs while reducing functional success, which means the agent is trading one failure mode for another instead of preserving both goals. Real guardrails sit outside the model and constrain what it can do, which repositories and services it can reach, and which automated checks must pass before code is accepted. Practical implication: treat policy enforcement and independent scanning as separate control layers, not as prompt hygiene.

Practical implication: treat policy enforcement and independent scanning as separate control layers, not as prompt hygiene.

NHI Mgmt Group analysis

AI-assisted coding has become a control-plane problem, not a productivity feature. Once agents can create and modify production code, they sit inside the same trust boundary as the systems they help build. That means insecure output is not a rare defect but an expected governance outcome unless the workflow is constrained. For identity and security programmes, the lesson is that code-generation systems need policy, approval, and verification controls just like privileged human users do. Practitioner conclusion: if the agent can change security-relevant logic, it must be governed as a high-risk actor.

Security debt grows when organisations confuse correctness with assurance. Passing tests, compiling successfully, or meeting a feature request does not mean the resulting code respects invariants such as session lifetime, URL validation, or secret handling. The article's findings align with broader secure development guidance in NIST CSF and NIST 800-53, where validation and change control are part of the control environment, not optional add-ons. Practitioner conclusion: tie release decisions to secure behaviour, not just developer acceptance.

Guardrails for AI-generated code should be designed like privileged access controls. The article implicitly describes a least-privilege problem, because agents need limited scope, explicit approvals, and independent oversight over what they can modify. That intersection matters to IAM and PAM teams, because AI coding agents increasingly act like ephemeral non-human identities operating across repositories, secrets, and deployment paths. Practitioner conclusion: apply the same discipline you would use for privileged automation, including bounded scope and auditable actions.

Deep remediation is where the security value is won or lost. Fixing one vulnerable line is not enough when an AI agent can spread similar mistakes across a codebase. The operational challenge is to detect patterns across files, validate the full fix, and ensure the next model run does not reproduce the same flaw. That is a governance problem as much as an engineering one, because it requires a repeatable control loop rather than ad hoc review. Practitioner conclusion: build remediation workflows that verify the whole change, not just the visible defect.

What this signals

AI-generated code changes the unit of control from developer intent to workflow assurance. Security teams should expect more code to arrive through semi-autonomous tools that require policy, review, and verification around the model rather than inside it. That means release governance will increasingly depend on independent checks, scoped permissions, and auditability of AI-assisted changes, especially where code touches authentication, secrets, or privilege boundaries.

Secret handling is where AI coding risk quickly becomes identity risk. AI-generated changes often intersect with credentials, tokens, and session logic, which are classic NHI governance domains. When the same workflow can create code and alter sensitive access paths, teams need controls that align with the NHI lifecycle and with secure development practices such as those discussed in the Ultimate Guide to NHIs , 2025 Outlook and Predictions.

Model behaviour must be governed as part of the software supply chain. The practical shift is to treat AI coding assistants as part of the build and change system, not as neutral tooling. For practitioners, that means policy enforcement, security validation, and remediation feedback loops need to sit between generation and merge, which is where OWASP Top 10 for Agentic Applications 2026 becomes operationally relevant.

For practitioners

• Gate AI-generated changes with security-specific acceptance criteria Require checks that evaluate exploitability, not only functional test pass rates, before merging code produced by agents. Use security assertions for invariants such as input sanitisation, session expiry, and header handling.
• Limit agent write access to constrained repository scopes Restrict which repos, branches, services, and secrets an AI coding agent can reach. Treat the agent as a privileged non-human actor with explicit boundaries and logged actions.
• Run independent scanners outside the model loop Separate detection from generation so the same agent that writes code is not the one validating its safety. Pair SAST, secret scanning, and policy checks with human review for high-risk diffs.
• Build deep remediation workflows for multi-file fixes When a vulnerability is found, verify that the fix addresses related files, helper functions, and surrounding logic rather than only the line flagged by the scanner. This reduces repeat exposure from the same underlying flaw.

Key takeaways

• AI coding agents can produce patches that satisfy tests while still leaving security defects intact.
• The main control gap is not model quality alone but the absence of independent guardrails, scoped permissions, and security-specific validation.
• Teams should govern AI-generated code like privileged automation, with least privilege, separate detection, and human approval for risky changes.

Key terms

• Vibe Coding: A software development approach where natural-language prompts drive much of the implementation and AI produces the code. In practice, the term covers a wide range of control levels, from no-review prototyping to structured engineering with tests, review, and architecture held by humans.
• Security Invariant: A condition that must remain true for software to be safe, such as safe input validation, constant-time comparison, or strict session expiry. AI-generated code often breaks invariants because it optimises for visible functionality rather than the hidden control logic that prevents abuse.
• Guardrails: Guardrails are policy controls that inspect prompts and model outputs against defined safety, privacy, and compliance rules. In AI operations, they reduce harmful language and disclosure risk, but they do not replace entitlement management, logging, or identity governance for the systems that call the model.
• Deep Remediation: A remediation approach that fixes the immediate defect and the surrounding code paths, helper functions, and related files that could reintroduce the same weakness. It is essential when AI-generated changes may replicate a bug pattern across multiple parts of a codebase.

What's in the full article

Symbiotic Security's full insights post covers the implementation detail this analysis intentionally leaves for the source:

• The full study breakdown of the 200-task benchmark design across 108 real GitHub repositories.
• The per-model and per-agent secure-pass comparisons that show where functional success diverges from secure success.
• The specific vulnerability classes observed in case studies, including timing leaks, CRLF injection, session lifetime flaws, and unsafe links.
• The guardrail patterns for MCP-aware workflows, including policy checks, independent detection, and deep remediation across files.

👉 Symbiotic Security's full post covers the benchmark design, failure patterns, and guardrail model in more depth.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and IAM. It is useful for practitioners who need to govern privileged automation, AI-enabled workflows, and identity risk across modern engineering programmes.