AI-generated code often works while remaining insecure in production

At a glance

What this is: This is an independent analysis of why AI-generated code can pass functional testing while still carrying serious security and access-control defects.

Why it matters: It matters because IAM, PAM, and broader security teams must now govern code generation, runtime permissions, and secrets exposure as part of the same risk chain.

By the numbers:

• Gartner warns that by 2028, prompt-to-app approaches adopted by citizen developers will increase software defects by 2500%.
• A Gartner survey of 175 employees conducted between May and November 2025 found that over 57% use personal GenAI accounts for work purposes.
• 33% admit inputting sensitive information into unapproved tools.

👉 Read OXSecurity's analysis of AI-generated code security failures and runtime blind spots

Context

AI-generated code changes the security problem from manual coding mistakes to machine-amplified pattern replication. The code may satisfy functional requirements, but that does not mean it enforces authentication, access control, validation, or safe dependency handling. For IAM teams, the relevance is clear: insecure application logic often creates downstream identity and secret exposure that no access review can repair after deployment.

The governance gap is that traditional AppSec assumes humans understand what they are shipping and can validate it with reviews, tests, and scanner output. In prompt-driven development, teams can deploy code they do not fully understand, which makes security validation dependent on runtime checks and adversarial testing rather than developer intuition. That is a broader security operations issue, but it also intersects with identity because compromised permissions, leaked tokens, and missing authorization are often the first exploitation path.

The article’s starting position is increasingly typical in AI-heavy development teams, not exceptional. The same pattern appears wherever speed, code generation, and incomplete security context converge.

Key questions

Q: What breaks when AI-generated code is reviewed without security gates?

A: What breaks is the assumption that a clean-looking diff is a safe diff. AI-generated code can hide unsafe defaults, dependency issues, and weak validation that basic review misses. Without CI-enforced tests and security checks, the merge process becomes a distribution channel for defects rather than a control point.

Q: Why do AI coding tools increase the risk of insecure authentication and authorisation?

A: They often reproduce common patterns from training data, including incomplete login flows, permissive access checks, and unsafe session handling. The model optimises for plausible code, not security intent. That makes identity controls especially vulnerable when developers accept generated output without understanding what must be enforced.

Q: How can security teams tell whether AI-generated code is actually safe?

A: They should look for security regression evidence, not just test pass rates. A codebase is safer only when the agent’s output survives threat-informed checks, review of privileged paths, and validation against the vulnerability classes most likely to recur in that repository.

Q: What should organisations do when AI-generated applications handle tokens or privileged access?

A: Treat those applications as part of the identity security surface. Review token scope, service-account use, database roles, and any code path that can create or consume credentials. If the app touches identity, IAM and PAM teams should validate the control design before release.

Technical breakdown

Why functional tests miss security defects in AI-generated code

Functional tests verify whether software behaves as requested under expected input. Security defects usually appear when input is hostile, permissions are mis-scoped, or a control exists in theory but not in runtime configuration. AI-generated code can therefore pass login, query, and workflow tests while still allowing injection, broken authorisation, or unauthorised data access. The issue is not only bad code. It is that the generation process optimises for visible behaviour, while security depends on invisible control enforcement.

Practical implication: teams need adversarial test cases and runtime validation, not just happy-path QA.

How AI models reproduce insecure patterns from training data

Large language models generate probable code patterns, not secure intent. Because training data contains legacy examples, deprecated cryptography, string concatenation, and incomplete auth flows, the model may reproduce those patterns when asked to build a feature quickly. This is why insecure dependencies, weak hashing, and direct user-input handling keep appearing in AI-generated code. The model does not recognise risk in the way a security engineer does; it predicts the next plausible token sequence.

Practical implication: security context must be embedded in prompts, guardrails, and code-generation workflows before output is accepted.

Why static analysis alone cannot prove runtime access control

Static analysis inspects code and dependencies in isolation. It cannot confirm whether a database has row-level security enabled, whether cloud permissions are actually effective, or whether a login flow blocks malicious parameter tampering. That matters because many AI-generated failures are configuration or behaviour gaps rather than syntax errors. A scanner may flag a secret or miss a dead code path, but it cannot independently verify that access restrictions are enforced end to end.

Practical implication: pair code scanning with behavioral testing that attempts unauthorised access against live controls.

Threat narrative

Attacker objective: The attacker wants to exploit functional-but-insecure AI-generated code to reach protected data, credentials, or privileged application paths.

1. Entry occurs when AI-generated code introduces insecure patterns such as exposed secrets, permissive endpoints, or vulnerable dependencies into an application or deployment pipeline.
2. Escalation follows when missing validation, weak authentication, or absent row-level controls lets an attacker move from exposed functionality to unauthorised data access.
3. Impact is data theft, credential exposure, or application compromise in systems that appeared to work correctly during normal testing.

NHI Mgmt Group analysis

AI-generated code creates security debt faster than conventional review can absorb. The central problem is not that AI writes bad code in every case, but that it writes plausible code faster than teams can validate it. That shifts risk from isolated defects to systemic governance failure. In that environment, application security becomes a control-plane problem that spans code generation, runtime permissions, and secret handling. Practitioners should treat AI-assisted development as a governance domain, not just a productivity choice.

Prompt-to-app workflows expose a new named concept: runtime security blind spots. These are defects that remain invisible until the code interacts with live data, active permissions, or real users. Static scanners can catch some issues, but they cannot prove that row-level security, authentication, or cloud permissions are actually working. This is where the identity bridge matters most: leaked API keys, permissive tokens, and missing authorisation all convert app defects into identity incidents. Teams need runtime assurance, not just code assurance.

Security validation must move into the generation loop if organisations want velocity without uncontrolled exposure. The article shows that waiting for CI/CD or post-build review is too late when insecure patterns are created at prompt time. That does not eliminate the need for human review. It does mean the review target changes from syntax and style to control enforcement, sensitive-data paths, and privilege boundaries. Practitioners should align AI coding governance with NIST Cybersecurity Framework 2.0 and security controls for access management, auditability, and configuration discipline.

The biggest risk sits in authentication and authorisation, not just in obvious code vulnerabilities. AI can produce working login flows that still leave bypasses, missing session controls, or unsafe access to data behind the interface. That is a classic identity governance failure in a new form. IAM and PAM teams should see AI-generated application logic as part of the access control surface, especially where tokens, service accounts, and database permissions are involved. The practical conclusion is simple: if the application can create or consume identity, identity governance must inspect it.

AI coding security is now a software supply chain issue as much as an application issue. The models can insert unverified dependencies, deprecated libraries, and insecure patterns directly into the codebase, which means the vulnerability enters before traditional dependency management has a chance to intervene. That makes software composition analysis necessary but not sufficient. The field should expect more control convergence between AppSec, IAM, and DevSecOps, because the same workflow now governs code, credentials, and runtime access. Practitioners should plan for security controls that operate before, during, and after code generation.

What this signals

Runtime security blind spots will become a recurring control failure wherever AI-generated code reaches production faster than reviewers can validate it. The operational shift is from scanning for defects to proving that authorisation, secrets handling, and input validation still hold under attack. For identity teams, that means the application layer now belongs in the access-governance conversation, not just the AppSec backlog.

The programme implication is straightforward: teams that keep treating AI coding as a developer productivity issue will keep finding security defects too late. Use the Guide to the Secret Sprawl Challenge to map how generated code, embedded secrets, and credential exposure connect across the build chain. Pair that with NIST Cybersecurity Framework 2.0 to anchor governance, detection, and response around the controls that actually matter.

Identity and application security are converging around the same failure mode. When code can create tokens, consume service accounts, or expose database access, the quality of prompt-time governance becomes an access-control problem. That is why teams should measure not only how much AI they use, but how often generated code reaches sensitive paths without human comprehension or runtime verification.

For practitioners

• Embed security context into prompt workflows Require generation prompts to include authentication, validation, secret-handling, and access-control requirements before code is accepted into the pipeline.
• Test live controls with adversarial cases Run behavioural tests against staging and production-like environments to confirm row-level security, token checks, and permission boundaries actually block unauthorised access.
• Treat generated dependencies as supply-chain risk Scan for vulnerable packages, transitive dependencies, and suspicious imports before build completion, then correlate findings with runtime reachability.
• Bring identity teams into AI code review Include IAM and PAM reviewers when AI-generated code creates or consumes tokens, service accounts, database roles, or privileged API paths.
• Use runtime validation for access-control claims Verify that claimed protections such as database row-level security, cloud policy enforcement, and session controls are active in the deployed environment, not just present in code.

Key takeaways

• AI-generated code can meet functional requirements while still failing basic security expectations, which makes normal testing an unreliable gate.
• The evidence points to widespread defects in injection protection, secret handling, and authorisation, especially when developers accept generated code without understanding it.
• The right control strategy combines prompt-time guardrails, runtime validation, and identity review wherever generated code handles credentials or access paths.

Key terms

• Prompt-to-App Platform: A tool that turns natural-language instructions into functional software, often with minimal manual coding. These platforms lower the barrier to entry, but they also create governance risk when generated output reaches production without architecture review, secret controls, or lifecycle management.
• Runtime security blind spot: A gap that only appears when software is running against live data, permissions, and user behaviour. Static tools can miss it because the code may look correct on paper while still failing to enforce row-level security, authentication, or other access controls in practice.
• Behavioural security testing: A validation approach that tries to break the application the way an attacker would, rather than only checking whether it functions as intended. It is especially important for AI-generated code because many failures arise from missing enforcement, not broken syntax or obvious implementation errors.
• Identity security surface: The set of application paths, services, tokens, and accounts that can create, consume, or protect identity. In AI-generated code, this surface expands quickly because the software may touch authentication, service accounts, database roles, and credential handling without the developer noticing the full impact.

What's in the full article

OXSecurity's full analysis covers the operational detail this post intentionally leaves for the source:

• Walkthroughs of specific AI coding failure modes across authentication, injection, and dependency selection.
• Examples of how AI-generated infrastructure and application code introduce hidden access-control gaps.
• Operational guidance on embedding security into prompting, review, and validation workflows.
• Additional discussion of how runtime testing can confirm whether access controls actually hold.

👉 OXSecurity's full post covers the code patterns, security gaps, and validation limits in more operational detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity controls to the broader security decisions that shape modern application risk.