AI supply chain risks are outpacing traditional AppSec controls

At a glance

What this is: This analysis argues that AI dependencies have become a hidden part of the software supply chain, and traditional AppSec controls do not fully inventory or govern them.

Why it matters: It matters because IAM, AppSec, and platform teams now have to account for AI services, agent frameworks, prompts, and MCP tooling as governed dependencies, not invisible runtime extras.

By the numbers:

• More than 75% of enterprises are already embedding LLMs, AI SDKs, and AI services directly into their applications.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read Checkmarx's analysis of AI supply chain risks in application security

Context

AI supply chain risk starts with a simple governance failure: organisations can pass conventional AppSec checks and still ship applications that call hosted LLMs, use agent frameworks, or pass prompts and embeddings through runtime services. That creates a new dependency layer that SBOM-era workflows do not reliably capture, which is why AI supply chain governance now sits alongside application security, secrets management, and identity controls.

The article's core point is that the problem is not only code vulnerability management. It is also control over what AI components exist, what they can reach, and who is accountable for them. That is where the identity intersection matters: hosted AI services, MCP servers, and agent frameworks often depend on credentials, tokens, and delegated permissions that are invisible if teams only review source code and package lists.

The starting position described here is increasingly typical, not exceptional. Many security programmes can validate code, but not AI assets, model provenance, or policy drift across AI-enabled dependencies.

Key questions

Q: What breaks when AI dependencies are not included in application security reviews?

A: Security teams lose visibility into a dependency class that can change behaviour, expose data, or invoke tools without appearing in a normal SBOM or vulnerability scan. That means clean SAST and SCA results can coexist with ungoverned AI services, agent frameworks, and runtime prompts. The result is a false sense of control, not real assurance.

Q: Why do AI agents create a governance problem for IAM teams?

A: AI agents create a governance problem because they authenticate and act as autonomous software entities with tool access. If their actions are logged only as application activity, teams lose accountability, context, and revocation clarity. IAM must therefore extend to agent identity, delegated authority, and control-plane audit trails.

Q: How do organisations know whether AI governance is actually working?

A: AI governance is working when teams can prove that data access, identity permissions, and runtime controls line up with policy in practice. A useful test is whether the organisation can answer who accessed what, through which identity, and whether any out-of-policy movement was blocked or detected in time.

Q: Who is accountable when unapproved AI dependencies ship into production?

A: Accountability should sit with the product and security owners who approved the release, but compliance and risk functions also need evidence that governance gates were defined and enforced. Frameworks such as the EU AI Act and ISO 42001 make the absence of traceable AI oversight a board-level problem, not just an engineering oversight.

Technical breakdown

Why AI dependencies evade traditional SBOM and SAST coverage

Traditional AppSec controls were built to enumerate code, open-source packages, containers, and known vulnerabilities. AI dependencies sit partly outside that model because they may be introduced through configuration, hosted services, prompts, or runtime orchestration rather than explicit code imports. An application can therefore change behaviour materially without a corresponding change in the visible software bill of materials. That is why AI-BOM concepts are emerging: they attempt to inventory model, framework, service, and policy dependencies together. The technical gap is less about scanning depth and more about dependency class mismatch.

Practical implication: extend software inventory to capture AI services, agent frameworks, prompts, embeddings, and external model endpoints.

How MCP and agent frameworks expand execution authority

Model Context Protocol, or MCP, lets AI agents connect to tools and data sources. That is useful operationally, but it also turns a model-driven workflow into an execution-capable system with broader reach than a static application dependency. If an MCP server is untrusted, shadowed, or misconfigured, the agent can inherit tool access that developers did not intend. Agent frameworks then amplify the issue because they coordinate tool selection and runtime decisions dynamically. The risk is not just data exposure. It is unauthorized action through delegated capability.

Practical implication: treat MCP servers and agent toolchains as privileged integrations and review them with the same care as other high-risk access paths.

What AI governance needs beyond vulnerability management

AI governance has to cover provenance, policy enforcement, and change control. A model can be safe in testing and unsafe under real prompts, while a provider can change retention terms or operational conditions without any code-level diff. That means governance must track model origin, approved registries, fine-tuning lineage, deployment context, and policy exceptions. Traditional vulnerability management can still matter, but it does not answer whether the AI component is authorised, traceable, or compliant. In practice, AI security becomes a control-plane problem as much as a code-scanning problem.

Practical implication: build approval and review gates for AI components, including model provenance checks and policy-based release controls.

Threat narrative

Attacker objective: The objective is to use hidden AI dependencies or untrusted AI supply chain components to gain broader access, manipulate behaviour, or expose sensitive data without detection.

1. Entry occurs when AI services, frameworks, prompts, or MCP servers are introduced into applications through code, configuration, or runtime dependencies that evade standard inventories.
2. Escalation happens when these components inherit broader execution or data access than development teams intended, especially where delegated credentials or tool permissions are reused.
3. Impact is driven by invisible AI behaviour changes, data exposure, or compliance drift that remain outside the scope of conventional AppSec validation.

NHI Mgmt Group analysis

AI supply chain governance is now an inventory problem before it is a scanning problem. Organisations cannot govern what they cannot enumerate, and AI assets often enter through runtime configuration, hosted APIs, and agent orchestration rather than visible source changes. That makes traditional AppSec evidence incomplete even when scans are clean. Practitioners should treat AI-BOM coverage as a prerequisite for control, not a reporting luxury.

Shadow AI dependencies create an identity and privilege problem, not just a model risk problem. Hosted LLMs, agent frameworks, and MCP servers often rely on credentials, tokens, and delegated permissions that sit outside existing review paths. When those access paths are unmanaged, the AI system becomes another non-human actor with effective privileges but weak governance. Teams should tie AI dependency review to IAM, PAM, and secret lifecycle controls, not leave it inside the AppSec queue.

AI governance debt is accumulating faster than most compliance teams can evidence it. The article rightly points to accountability pressure from EU AI Act and ISO 42001 style governance expectations, but the operational issue is earlier in the chain: provenance, policy drift, and runtime change are not being documented consistently. That leaves organisations unable to prove what AI is running, where it came from, or what it can do. Practitioners should build audit-ready AI change records now, before the reporting burden becomes regulatory debt.

Model-boundary controls are becoming the new control surface for application security. Traditional AppSec assumed the application boundary was the codebase and its packages. In AI-enabled software, the effective boundary extends to prompts, embeddings, vector stores, model endpoints, and tool connectors. That boundary shift complicates secure development, but it also creates a chance to unify AI governance with NIST AI RMF, OWASP Agentic AI Top 10, and existing software supply chain practices. Teams should align controls to the dependency boundary, not the repository boundary.

What this signals

AI supply chain governance is converging with identity governance because hosted models, agent frameworks, and MCP servers depend on credentials and delegated permissions just like other non-human systems. That means security leaders should expect AI inventory, secret lifecycle, and access approval workflows to become part of the same control conversation, especially where runtime services can act outside the repository boundary.

Model-boundary drift: the useful security boundary is no longer the codebase alone, but the full set of AI endpoints, prompts, and tool connectors that can alter application behaviour. Security programmes that continue to optimise only for static software artefacts will miss the governance layer where most AI risk now accumulates. Teams should prepare for AI asset attestation to become a routine control expectation.

Regulatory pressure will force the issue even where engineering urgency does not. Once organisations have to evidence model provenance, policy enforcement, and change control, the gap between AppSec validation and AI governance becomes visible to auditors, not just architects. That is why AI-BOM work should be treated as a programme capability, not an isolated tooling project.

For practitioners

• Inventory AI dependencies across the release path Map hosted model calls, AI SDKs, agent frameworks, prompts, embeddings, vector stores, and MCP servers into the same asset view used for application and secret inventories.
• Classify AI components as governed dependencies Require approval for any AI service or model endpoint that can change application behaviour, access data, or invoke tools, and route it through release governance.
• Tie AI access to secret and identity controls Review API keys, tokens, service accounts, and delegated permissions used by AI workloads under the same lifecycle and least-privilege rules applied to other non-human identities.
• Add AI-specific checks to policy gates Block releases that introduce unapproved models, shadow MCP servers, or undocumented policy drift, and require evidence of provenance and retention terms before deployment.
• Create audit-ready AI change records Track model origin, fine-tuning lineage, approved registries, and runtime exceptions so compliance teams can demonstrate what AI is running and who approved it.

Key takeaways

• AI supply chain risk is a governance gap first and a scanning gap second, because clean AppSec results can still hide AI dependencies.
• Hosted LLMs, agent frameworks, prompts, and MCP servers create non-human access paths that need identity and lifecycle controls.
• The practical response is AI inventory, policy enforcement, and audit-ready provenance tracking, not reliance on traditional vulnerability management alone.

Key terms

• AI-BOM: An AI bill of materials is a structured inventory of the components that define an AI agent, including the model, prompt, tools, retrieval sources, and dependencies. In practice, it is the evidence base for review, change control, and risk assessment when the agent evolves after deployment.
• MCP Server: An MCP server is a tool endpoint that connects an AI agent to external systems and data sources through Model Context Protocol. Because it extends what the agent can reach, it becomes part of the identity and access surface and must be reviewed like any other privileged connector.
• Shadow AI dependency: A shadow AI dependency is an AI service, model, framework, or agent component that is used in production but not captured in formal inventories or approval workflows. These dependencies create governance blind spots because they can affect behaviour, access, and compliance without being visible to standard AppSec controls.
• Model Provenance: Model provenance is the evidence chain showing where an AI artefact came from, how it was modified, and whether the version in use is the one that was approved. For AI security teams, provenance is the control that turns trust from assumption into verification.

What's in the full article

Checkmarx's full article covers the operational detail this post intentionally leaves for the source:

• The 10 AI supply chain risk categories and how they map to real dependency patterns in application pipelines.
• The practical AI Supply Chain Maturity Model for moving from unknown exposure to governed control.
• A side-by-side comparison of traditional SBOMs versus AI-BOMs for inventory and compliance work.
• The two-floor security architecture that separates what to preserve from existing AppSec and what to add for AI dependencies.

👉 Checkmarx's full article covers the AI Supply Chain Maturity Model, AI-BOM comparison, and control architecture details.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management in a way that supports broader identity and access decisions. It gives security practitioners a practical baseline for managing non-human access across modern programmes.