TL;DR: AI-generated code produces 1.7x more vulnerabilities than human-written code, while most organisations cannot inventory embedded models, MCP servers, agent frameworks, datasets, or prompts, according to Checkmarx. The governance problem is no longer just code volume but unseen AI components that create new supply chain attack paths and audit gaps.
At a glance
What this is: Shadow AI is expanding the software supply chain beyond code into models, agents, datasets, MCP servers, and prompts that conventional AppSec inventory controls often miss.
Why it matters: AppSec and supply chain teams need a deterministic way to discover and govern AI components because invisible assets create both security exposure and compliance gaps.
Context
Modern software supply chains now include AI assets as well as code. That matters because traditional AppSec tooling was built to inventory libraries, packages, and configurations, not models, agent frameworks, datasets, prompt files, or MCP servers embedded in development workflows.
The governance gap is visibility, not just volume. When teams cannot reliably answer what AI assets exist, where they are embedded, and what they do, they lose the ability to review risk, enforce policy, or support audit-ready evidence across the development lifecycle.
Key questions
Q: What breaks when AI components are invisible to AppSec inventory controls?
A: When AI components are invisible, teams cannot tell which models, agents, datasets, or prompts are actually in use, so policy, risk review, and exception handling all become incomplete. The practical failure is governance without evidence. Security teams lose the ability to enforce consistent controls before risky AI assets are embedded in code and CI/CD workflows.
Q: Why do AI supply chain risks create governance exposure even when code scans are clean?
A: Code scans only cover software defects, not model behaviour, runtime prompts, third-party terms, or tool access. That is why a clean SAST or SCA result does not prove the AI layer is safe, compliant, or within policy boundaries.
Q: What are the signs that shadow AI controls are failing in practice?
A: Shadow AI controls are failing when sensitive data moves through copy-paste, uploads, or API calls outside sanctioned workflows, yet the organisation has no alerting or enforcement at the point of transfer. Other warning signs include heavy personal account use, unclear approved-tool lists, and developers submitting source code or internal structures to external AI services without review.
Q: How should security teams include AI applications in software supply chain security programs?
A: Security teams should treat AI applications as part of the software supply chain, not as a separate innovation track. That means enforcing secure development practices, validating data provenance, reviewing third-party models and libraries, and monitoring deployments continuously. The goal is to reduce exposure from tampering, malicious inputs, and ungoverned dependencies before AI systems reach production.
Technical breakdown
Why AI assets fall outside standard dependency graphs
Traditional software composition analysis is designed around package manifests, transitive libraries, and build artefacts. AI-enabled applications add a parallel layer made up of model references, embeddings, datasets, prompts, agent frameworks, and MCP servers, many of which live in configuration files or source code rather than dependency metadata. That means the usual inventory model misses the things that actually shape runtime behaviour and supply chain exposure. Deterministic discovery has to look at imports, manifests, file paths, and configuration signals, not infer presence from heuristics. Practical implication: teams need discovery that treats AI components as first-class software assets, not as incidental code patterns.
Practical implication: move discovery to code and configuration signals so AI assets are inventoried before they reach review or release.
How shadow AI creates supply chain risk in AppSec pipelines
Shadow AI is the uncontrolled adoption of models, tools, and agent workflows without formal review. Once these assets enter the codebase, they can introduce model poisoning, unverified weights, unsafe autonomous tool invocation, unpinned versions, and exposed datasets or embeddings. The risk is compounded because these components can alter application behaviour without looking like traditional dependency changes. In practice, this turns AppSec from a code-quality gate into a supply chain governance function for AI artefacts. Practical implication: policy needs to evaluate AI assets at the point of introduction, not after they are already embedded in pipelines and applications.
Practical implication: enforce policy on AI component intake, not only on code findings after the build has already progressed.
Why probabilistic AI detection is weak for audit and governance
Some tools try to identify AI content by inference, but that creates non-deterministic results that are hard to defend to auditors. If discovery cannot be traced back to a specific line of code, manifest entry, or configuration item, then the organisation cannot prove what it found or why it made a given risk decision. That weakens compliance reporting and makes policy enforcement inconsistent across teams and repositories. For governance, the standard is evidence, not likelihood. Practical implication: use detection that produces traceable artefacts and supports a repeatable inventory of models, agents, datasets, and prompts.
Practical implication: require evidence-backed findings that can be traced to source artefacts and repeated consistently across scans.
Threat narrative
Attacker objective: The objective is to exploit unmanaged AI components to reach application behaviour, data, or downstream tooling through a supply chain path that normal AppSec controls do not fully see.
- Entry occurs when developers import public models, agent frameworks, MCP servers, or prompt libraries into applications without formal governance.
- Credential and trust boundaries are then extended to AI components that can invoke tools, access datasets, or change application behaviour outside the normal dependency review path.
- The impact is a widened software supply chain attack surface, with opportunities for model poisoning, unsafe autonomous actions, and exposure of sensitive data or embedded prompts.
Breaches seen in the wild
- LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.
- Hugging Face Spaces breach 2024: Unauthorised access to Hugging Face Spaces may have exposed secrets users stored for AI apps; tokens were revoked and org tokens removed.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Shadow AI is now a supply chain governance problem, not just an AppSec extension. The article shows that the asset boundary has expanded beyond code and packages to models, prompts, datasets, agent frameworks, and MCP servers. That means the control question is no longer only whether a build passed scan, but whether the organisation can inventory the actual components that shape runtime behaviour. Practitioners should treat AI component discovery as part of core software governance.
Deterministic discovery is the decisive control shift. When AI components are inferred probabilistically, auditors cannot reliably reconstruct what was present at scan time or why a risk flag appeared. Code-first detection anchored in manifests, imports, and configuration creates the evidence trail required for governance, exception handling, and consistent policy enforcement. The broader market is moving toward proof of presence rather than guesswork, and that is the right direction for AI supply chain control.
AI-BOM discipline will become a baseline requirement for governed AI development. The article’s focus on models, datasets, licenses, and embedded prompts shows that the bill of materials has to cover AI artefacts, not just open-source packages. That changes how security, engineering, and compliance teams talk to each other because the inventory now has operational, legal, and risk dimensions at once. Practitioners should expect AI-BOM coverage to become a standard evidence layer in software governance.
AppSec programs that only measure vulnerability volume will miss the real governance gap. The article notes both higher vulnerability density in AI-generated code and the growth of unseen AI assets, which together create backlogs and blind spots. The important issue is not simply more findings, but more unreviewed attack paths entering faster than teams can evaluate them. Security leaders should judge their programmes on visibility and control over AI inputs, not scan counts alone.
From our research library:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
- 96% of security operations teams report critical blind spots, most commonly in cloud infrastructure (74%) and identity and access behaviour (67%).
- Read next: AI Supply Chain Security and AI-BOM Guide
What this signals
Shadow AI expands the unit of governance. AppSec teams are no longer only protecting code paths and open-source dependencies. They now have to govern model references, dataset inputs, agent frameworks, and MCP servers as part of the same delivery system, which makes inventory quality a first-order control issue.
AI-BOMs will become operational evidence, not documentation. Once AI assets are being shipped alongside conventional dependencies, the bill of materials has to support policy enforcement, audit queries, and exception management. Organisations that cannot tie AI artefacts to source evidence will struggle to defend their security decisions to auditors or developers.
Visibility into identity and access matters as these stacks mature. As AI systems start invoking tools and pulling sensitive data, the access model behind those components becomes part of the control surface. That is why machine identity, approvals, and governance over privileged access to AI services will matter more as shadow AI becomes normalised.
For practitioners
- Implement code-first AI asset discovery Inventory models, agent frameworks, MCP servers, datasets, embeddings, and prompt files directly from source and configuration instead of relying on dependency manifests alone.
- Add AI intake policy gates Require approved model lists, framework allowlists, version pinning, and review of dataset references before AI components are merged into production paths.
- Extend AppSec workflows to AI artefacts Surface AI findings in the same pull request, CI/CD, and dashboard workflows used for SAST and SCA so teams do not create a parallel review process.
- Document evidence for audits Preserve the exact code and configuration locations that prove each AI asset exists so compliance teams can explain inventory and risk decisions without inference.
Key takeaways
- AI-enabled software now includes assets that traditional AppSec inventory controls were never designed to see, which creates a governance gap before a vulnerability even exists.
- The article’s evidence points to a dual problem of higher vulnerability volume and lower visibility, which together overwhelm scan-centric programmes.
- Teams that want control need deterministic discovery, evidence-backed AI inventories, and policy gates that operate at intake rather than after deployment.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 and MITRE ATT&CK address the attack surface, NIST CSF 2.0 and NIST AI RMF set the technical controls, and EU AI Act defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-03 — Vulnerable Third-Party NHI | Public models, agent frameworks, and MCP servers introduce third-party AI assets into the supply chain. |
| NHI-02 — Secret Leakage | AI prompts, datasets, and embedded credentials can be exposed in source and configuration files. | |
| Recommendation — Inventory and review third-party AI assets before they enter build and runtime paths. Scan code and configuration for exposed AI-related secrets and remove them before release. | ||
| MITRE ATT&CK | TA0006;TA0007 — Credential Access; Discovery | Shadow AI supply chain abuse depends on discovering hidden assets and reaching sensitive data or tooling. |
| Recommendation — Map AI supply chain findings to discovery and credential-access tactics to prioritise hunts. | ||
| NIST CSF 2.0 | ID.AM-02 — Software, hardware, data, personnel, devices, and facilities are inventoried | The article’s core gap is incomplete inventory of software and AI assets. |
| PR.AA-05 — Access Permissions, Entitlements and Authorizations | AI agents and toolchains can reach data and services through overbroad authorisation. | |
| Recommendation — Extend asset inventory controls to include models, datasets, prompts, and agent frameworks. Apply entitlement controls to AI services and agents before they can invoke tools or data sources. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article centres on governance of AI assets across the software supply chain. |
| Recommendation — Define ownership and accountability for AI assets across development, security, and compliance functions. | ||
| EU AI Act | Art. 9 — Risk management system | The article explicitly ties AI component governance to regulatory readiness. |
| Recommendation — Maintain a documented AI risk management process for models, datasets, prompts, and agent workflows. | ||
Key terms
- Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
- AI bill of materials: An inventory of the components used by an AI development environment, including models, plugins, hooks, skills, and agent integrations. It gives security teams visibility into what is operating in the stack so they can assess trust, dependency risk, and governance coverage.
- Deterministic discovery: Deterministic discovery is a method of identifying AI assets from explicit code and configuration signals such as imports, manifests, file paths, and settings. It is preferred for governance because it creates repeatable, auditable evidence rather than relying on probabilistic inference.
- AI Supply Chain Security: AI supply chain security is the practice of protecting the people, data, models, code, tools, and services used to build and run AI systems. It covers risks from training data, model artifacts, dependencies, prompts, connectors, and deployment pipelines, with controls for integrity, provenance, access, and change monitoring across the full lifecycle.
Deepen your knowledge
NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management for practitioners who need to manage access and trust across modern delivery environments. It helps security and identity teams connect governed identity practices to the operational realities of AI and software supply chains.
Published by the NHIMG editorial team on August 1, 2026.
Updated on October 13, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org