Notifications
Clear all
Topic starter
19/08/2026 3:45 pm
TL;DR: AI coding agents can pass functional tests while still shipping vulnerable code, according to Symbiotic Security research, with the best agent-model pairing succeeding functionally on 61% of tasks but securing only 10.5% end to end. The control gap is not prompt quality but system-level guardrails, independent checks, and human approval around AI-assisted development.
NHIMG editorial — based on content published by Symbiotic Security: Vibe Coding Is Not Secured by Default: What a New Study Tells Us About AI-Generated Code
By the numbers:
- About 75% of developers already vibe code, according to the article's cited survey.
Questions worth separating out
Q: How should teams govern AI agents that refactor production systems?
A: Teams should govern agentic refactoring as a constrained execution problem, not a free-form coding problem.
Q: Why do functionally correct AI patches still create security risk?
A: Because a patch can satisfy tests while still violating the security invariant the original bug depended on.
Q: What do security teams get wrong about AI guardrails and prompt refusal?
A: They treat refusal as if it were a security boundary.
Practitioner guidance
- Gate AI-generated changes with security-specific acceptance criteria Require checks that evaluate exploitability, not only functional test pass rates, before merging code produced by agents.
- Limit agent write access to constrained repository scopes Restrict which repos, branches, services, and secrets an AI coding agent can reach.
- Run independent scanners outside the model loop Separate detection from generation so the same agent that writes code is not the one validating its safety.
What's in the full article
Symbiotic Security's full insights post covers the implementation detail this analysis intentionally leaves for the source:
- The full study breakdown of the 200-task benchmark design across 108 real GitHub repositories.
- The per-model and per-agent secure-pass comparisons that show where functional success diverges from secure success.
- The specific vulnerability classes observed in case studies, including timing leaks, CRLF injection, session lifetime flaws, and unsafe links.
- The guardrail patterns for MCP-aware workflows, including policy checks, independent detection, and deep remediation across files.
👉 Read Symbiotic Security's analysis of why vibe coding is not secured by default →
AI-generated code security gaps: are your guardrails enough?
Explore further
19/08/2026 4:22 pm
AI-assisted coding has become a control-plane problem, not a productivity feature. Once agents can create and modify production code, they sit inside the same trust boundary as the systems they help build. That means insecure output is not a rare defect but an expected governance outcome unless the workflow is constrained. For identity and security programmes, the lesson is that code-generation systems need policy, approval, and verification controls just like privileged human users do. Practitioner conclusion: if the agent can change security-relevant logic, it must be governed as a high-risk actor.
A question worth separating out:
Q: What should teams do before trusting AI output on code findings?
A: Teams should narrow the problem with reachability analysis, taint flow checks, and context about how the application is actually used. That reduces hallucination-like overreach and keeps the model focused on evidence rather than guesses. The best AI results come from bounded inputs, not from scanning the entire codebase blindly.
👉 Read our full editorial: Vibe coding leaves AI-generated code secure by exception
