AI coding assistants widen appsec gaps across code and supply chain

At a glance

What this is: AI coding assistants are accelerating software delivery, but the article argues they also increase insecure code, supply chain exposure, and review blind spots in AppSec workflows.

Why it matters: This matters because engineering leaders need to govern AI-produced code as a security input, not just a productivity feature, with controls that catch vulnerable patterns, phantom packages, and weak review coverage.

By the numbers:

• 84% of developers now use or plan to use AI coding assistants, and 51% rely on them daily.
• GitGuardian found that 6.4% of repositories using GitHub Copilot leak at least one secret.
• 9% of respondents consider AI-driven security analysis a, er AI-driven security analysis a must-have capability.

👉 Read Kusari's analysis of AI coding assistant risk and AppSec controls

Context

AI coding assistants have changed the shape of application security because they increase the volume of code that reaches review while reducing the time teams spend reviewing it. The primary governance gap is not whether developers should use these tools, but whether security controls can keep pace with AI-generated code, hallucinated dependencies, and weaker defensive programming across the software development lifecycle.

In identity-adjacent terms, this is an assurance problem: the code-producing system is not a human user, yet it can still inject secrets, privilege paths, and transitive supply chain risk into production workflows. That makes AppSec closer to identity governance than many teams assume, especially when AI-generated changes touch authentication, authorization, secrets handling, and build pipelines.

Kusari's framing is typical of the current market: adoption is broad, the productivity benefit is real, and the security control model is still catching up. The article therefore reflects a common enterprise condition rather than an edge case.

Key questions

Q: How should security teams implement controls for AI-generated code in pull requests?

A: Security teams should make pull request scanning the default control point for AI-generated code. Combine SAST for insecure logic and SCA for dependencies, then block merges when the pipeline finds severe issues. The goal is to move review from informal human judgement to enforceable policy before code reaches the main branch.

Q: Why do AI coding agents increase supply-chain risk?

A: AI coding agents can choose tooling, install packages, and edit lock files in ways that may bypass the controls humans expect in a managed environment. That widens the blind spot around untracked software and makes package provenance harder to enforce. The practical response is to restrict agent permissions, inventory their workspaces, and review their install behaviour like any other privileged automation.

Q: What do teams get wrong about AI-generated documentation and code review?

A: They often assume documentation or review output is proof of oversight. In practice, if AI generates the work and another AI validates it, the process can become a closed loop unless a separate human applies challenge, context, and responsibility for the final decision.

Q: Should organisations treat AI coding assistants as part of the security boundary?

A: Yes. If an AI assistant can generate code, open pull requests, or influence deployment, it is operating inside the security boundary and must be governed accordingly. That means controlling its connectors, monitoring its outputs, and limiting the credentials it can access.

Technical breakdown

Why AI-generated code reproduces insecure patterns

Large language models predict plausible code by learning from massive training corpora, not by applying security policy. If vulnerable constructs such as string-concatenated SQL, weak input handling, or insecure deserialisation appear frequently in source material, the model can reproduce them because they are statistically common, not because they are safe. This is why AI output often looks correct while still violating secure coding norms. The problem is amplified when developers accept the first generated version without a security review loop.

Practical implication: Treat AI-generated code as untrusted input and force automated security validation before merge.

How slopsquatting turns hallucinated dependencies into supply chain risk

LLMs sometimes invent package names that do not exist, especially when asked for libraries or examples in unfamiliar ecosystems. Attackers can register those invented names in public registries, turning model hallucinations into a supply chain attack path known as slopsquatting. The risk is not limited to obviously malicious prompts because repeated hallucinated names can recur across queries, which makes them predictable targets. Traditional dependency scanning finds known vulnerabilities, but it does not by itself detect phantom packages or newly created malicious lookalikes.

Practical implication: Add dependency allowlisting and registry verification for every new package suggestion from AI tools.

Why missing defensive controls matter as much as visible bugs

AI-generated code often omits security logic that a human engineer would normally add, such as input validation, sanitisation, or explicit authorisation checks. That omission happens because the prompt optimised for function, not control coverage. In practice, this creates code that passes a basic functional test while still exposing attack surfaces in endpoints, pipelines, and libraries. Security review must therefore check not only for visible vulnerabilities, but also for absent safeguards that should have been present in the first place.

Practical implication: Map review gates to missing controls, not just known weakness signatures, so omitted validation is caught early.

Threat narrative

Attacker objective: The objective is to turn AI-assisted development speed into a durable application or supply chain weakness that can be exploited at scale.

1. Entry begins when an engineer accepts AI-generated code or package suggestions inside the development workflow, creating a path for insecure logic or phantom dependencies to enter the repository.
2. Escalation follows when those suggestions are merged without strong PR-level security review, allowing vulnerable patterns, malicious packages, or leaked secrets to move into build and deploy pipelines.
3. Impact occurs when the resulting application ships with exploitable code, exposed credentials, or compromised supply chain dependencies that attackers can reuse for intrusion or exfiltration.

NHI Mgmt Group analysis

AI-assisted development has become an application security governance problem, not just a developer productivity choice. Once AI tools are part of the code path, organisations inherit a second code author whose output must be controlled like any other untrusted supply chain input. That changes how security, engineering, and platform teams should think about code review, dependency trust, and release gating. The practical conclusion is that AppSec controls must sit inside the workflow, not outside it.

Slopsquatting is the clearest named concept emerging from this category of risk. Hallucinated packages create a repeatable attack surface because attackers can register names that models invent. This is a governance failure as much as a detection failure, because teams that do not verify package provenance have effectively delegated part of the supply chain to probabilistic output. Practitioners should treat package suggestion as a trust decision, not a convenience feature.

Security review is failing when it checks for known bugs but not for missing defensive intent. AI output can be functionally correct while still lacking validation, authentication, or authorisation logic that would normally be standard. That is where the control gap sits: the system is producing code that can compile, but not code that can be safely deployed. Teams should shift review criteria toward control completeness and secure-by-default patterns.

Identity intersects here because code, secrets, and access paths now move together. AI-assisted changes frequently touch authentication flows, token handling, and service credentials, which means weak AppSec controls can become identity control failures. The broader lesson for IAM and NHI teams is that software delivery pipelines now influence identity risk directly, so identity governance has to extend into build and review systems.

The market is moving toward integrated guardrails inside the developer workflow, not isolated scanning after the fact. The article reflects a broader shift away from point tools that only report findings and toward controls that evaluate code, dependencies, and secrets before merge. For practitioners, that means the security model must become continuous, inline, and tied to release decisions rather than periodic inspection.

What this signals

AI coding assistants are changing the pace of application change faster than most governance models can absorb. The operational signal for practitioners is that code review capacity, dependency trust, and secret handling all need to move earlier in the delivery process. Teams that still rely on periodic security review will continue to accumulate avoidable risk across release pipelines and authentication-heavy services.

Slopsquatting should now be treated as a control design issue, not a niche research finding. Once hallucinated packages become predictable, package provenance and registry policy must become first-class guardrails in developer platforms. For teams that manage credentials or service identities in code, this also means build systems and secret stores need tighter coupling, because dependency trust and identity trust are increasingly the same problem.

The broader signal is that AppSec is becoming a continuous identity-adjacent control plane. When AI tools write code that creates, moves, or handles secrets, the boundary between software security and identity governance narrows. That is why practitioners should align PR controls with NIST Cybersecurity Framework 2.0 and review AI-related risks through the lens of secure development practices, provenance, and release assurance.

For practitioners

• Treat AI-generated code as untrusted input Require the same level of security scrutiny for AI-assisted changes as for code from any external contributor, especially where authentication, validation, or secrets handling are involved.
• Embed SAST and SCA in the pull request workflow Run static application security testing and software composition analysis automatically on every PR so insecure patterns and transitive dependency issues are visible before merge.
• Block unverified package names from AI suggestions Add registry checks and allowlists so hallucinated dependencies, typosquats, and newly registered lookalikes cannot move from suggestion into the build.
• Review for missing controls, not just known vulnerabilities Make reviewers check for omitted input validation, authorisation, sanitisation, and secure defaults when AI tools generate endpoints or service logic.
• Tie code review to secret exposure detection Use inline secret scanning and credential hygiene checks in the same workflow that accepts AI-assisted changes, because secret leakage is one of the fastest paths from code generation to compromise.

Key takeaways

• AI coding assistants are accelerating code production faster than most application security controls can inspect it.
• The most material risks are insecure inherited patterns, hallucinated dependencies, and missing defensive logic, not just obvious syntax errors.
• Teams should move security validation into the pull request workflow and verify every AI-suggested dependency before merge.

Key terms

• AI coding assistant: An AI coding assistant is software that helps write, refactor, debug, or navigate code using model-driven suggestions and sometimes command execution. In practice, it can become part of the operational control plane if it has access to files, terminals, and external tools that can change a system.
• Slopsquatting: Slopsquatting is a supply-chain attack that exploits hallucinated package names suggested by AI systems. An attacker registers the invented name in a public registry and waits for a developer or build pipeline to install it. The risk sits at the intersection of model error, dependency trust, and software delivery speed.
• Transitive Dependency: A dependency brought in indirectly by another package rather than chosen explicitly by the developer. These nested components often escape casual review, which is why software composition analysis must cover the full dependency tree, not only top-level libraries.
• Defensive Programming: Code practices that assume inputs, states, and dependencies may be hostile or malformed. In this context, it includes validation, sanitisation, and authorisation checks that AI-generated code may omit unless the prompt or review process explicitly requires them.

What's in the full article

Kusari's full article covers the operational detail this post intentionally leaves for the source:

• Feature-by-feature breakdown of AI code review and dependency management capabilities in the developer workflow
• Survey table showing how developers rank SAST, SCA, IDE integration, and AI security analysis priorities
• Practical examples of inline PR security analysis for direct and transitive dependencies
• Discussion of how maintainers used the tool to catch issues that AI code generation missed

👉 Kusari's full article covers the developer survey data, control workflow examples, and AI code review gaps in more detail.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It is a practical fit for security practitioners who need to connect identity controls to modern software delivery risk.