AI inventory gaps are turning software supply chains blind

At a glance

What this is: AI assets are already embedded in software supply chains, and the key finding is that most organisations cannot reliably inventory or govern them.

Why it matters: This matters to IAM and security teams because untracked models, MCPs, agents, and libraries introduce identity, provenance, and access-control blind spots that existing supply-chain and AppSec processes do not fully cover.

By the numbers:

• 60% of their organisation’s code is written by, r organisation’s code is written by AI.
• Only 18% have any sort of AI governance in place.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.
• 80% of organisations report their AI agents have already performed actions beyond their intended scope.

👉 Read Checkmarx’s analysis of AI assets in the software supply chain

Context

AI supply chain risk is the gap between what developers embed in code and what security teams can actually inventory, review, and control. In this case, the primary keyword is AI supply chain security, and the issue is not theoretical: models, weights, MCPs, agent frameworks, and AI libraries are becoming part of production software before governance catches up.

The governance failure is familiar to identity teams. When software components can access data, tools, and workflows without clear ownership, provenance, or policy enforcement, they behave like unmanaged non-human identities inside the development lifecycle. That creates a control problem across AppSec, IAM, PAM, and software supply-chain governance, which is typical of organisations moving fast on AI adoption and slow on inventory discipline.

Key questions

Q: How should security teams build an AI inventory that is actually governable?

A: Start with a single authoritative record that captures the model or tool, the business use case, the dataset lineage, the owning team, the deployment environment, and the current governance status. Then connect that record to approval workflows, access reviews, and risk thresholds so the inventory can drive action instead of just reporting.

Q: Why do AI supply chains create identity and access risk?

A: Because AI systems rely on service accounts, API keys, federation paths, and delegated tool permissions. Those identities can read data, call services, or trigger actions, so weak lifecycle management creates hidden trust paths that security teams often fail to inventory or revoke quickly.

Q: What do organisations get wrong about AI governance and identity controls?

A: They often separate AI governance from identity governance, even though AI systems can shape access, code, and security decisions. That split leaves approval paths, accountability, and secret handling under-specified. A workable programme treats AI participation as part of the identity control problem.

Q: How can teams reduce risk from AI-generated code in supply chains?

A: Teams should apply the same provenance and review expectations to AI-generated code as they do to third-party dependencies. That means verifying source, tightening review on generated changes, and blocking untrusted artifacts from reaching deployment. The goal is to prevent automation from scaling uncertainty faster than governance can absorb it.

Technical breakdown

What the AI inventory gap actually is

The AI inventory gap is the absence of a trustworthy record of which AI components exist across code, pipelines, containers, and services. It includes models, datasets, prompts, agent frameworks, MCP servers, and AI libraries that are shipped as dependencies but never formally reviewed. Unlike a conventional package inventory, this must also capture provenance, versioning, data exposure, and runtime behaviour. Without that, teams are blind to what is executing, what it can reach, and which governance controls should apply.

Practical implication: build a deterministic inventory that tracks AI assets by type, provenance, version, and runtime location.

Why MCPs and agent frameworks change the risk model

MCPs connect models to tools and data sources, while agent frameworks orchestrate autonomous behaviour across systems. That means an AI component can move from passive code dependency to active decision-maker with permissions, context, and external reach. From a security standpoint, this is where identity and privilege become central. If the asset can call tools, read data, or trigger workflows, it needs governance comparable to other high-trust non-human identities, not just software package review.

Practical implication: treat tool-connected AI components as privileged identities and define access boundaries before deployment.

Why traditional AppSec scanners are not enough

Traditional scanners are good at code patterns, vulnerable dependencies, and known bad inputs, but they do not solve provenance, ownership, or policy enforcement for AI assets. An organisation can detect prompt-injection risk in code and still have no idea which model is embedded in a container image, whether its weights were verified, or whether a dataset contains licensed or sensitive content. The control gap is governance coverage, not just detection coverage. That is why AI supply chain security needs inventory, policy, and enforcement together.

Practical implication: pair detection tools with enforceable policy gates for model, dataset, and dependency approval.

Threat narrative

Attacker objective: The attacker objective is to introduce or exploit ungoverned AI assets that can exfiltrate data, alter behaviour, or expand access inside the software supply chain.

1. Entry occurs when developers embed models, MCPs, datasets, or agent frameworks into repositories or container images without security review or approval.
2. Escalation follows when those AI assets are allowed to access files, networks, prompts, or workflows with undocumented permissions and no provenance checks.
3. Impact arrives when poisoned, malicious, or over-privileged AI assets create data exposure, workflow abuse, compliance failure, or downstream supply-chain compromise.

NHI Mgmt Group analysis

AI supply chain security is becoming an identity problem, not just an AppSec problem. Once a model, MCP, or agent framework can access tools and data, it behaves like a non-human identity with privileges that need lifecycle control. That shifts the governance question from code scanning to ownership, scope, and revocation. Organisations that keep treating these assets as passive dependencies will miss the access risk embedded in runtime AI behaviour.

AI inventory gap: this is the control failure this article exposes, and it is more precise than general “shadow AI” language. The real issue is that enterprises cannot answer what AI assets exist, where they run, or who is accountable for them. That breaks provenance, auditability, and policy enforcement at the same time. In governance terms, the absence of an inventory is the absence of control, and that is the condition attackers and compliance failures exploit.

Software bills of materials are necessary but no longer sufficient. A BOM can list components, but AI assets also require provenance, weights integrity, dataset lineage, and runtime permission boundaries. Without those dimensions, a standards-aligned inventory still leaves operational blind spots. The practical conclusion is that AI governance has to extend beyond dependency management into enforceable trust boundaries across development and deployment.

Ownership fragmentation is the hidden reason AI governance lags. The article describes a familiar pattern: developers optimise for delivery, AppSec documents after the fact, and leadership lacks a single accountable owner. That same fragmentation appears in identity programmes when no one owns machine or workload identity end to end. The lesson for practitioners is to assign one governance authority before AI asset sprawl becomes permanent.

Policy needs to sit inside the delivery pipeline if AI risk is to be governable. Inventory created after release is too late for models, prompts, and agents that can change behaviour overnight. Governance has to move into PR checks, CI/CD gates, and runtime policy enforcement so that approval, version control, and revocation happen where the asset enters the environment. The discipline is closer to identity governance than to traditional vulnerability management.

What this signals

AI inventory will become a programme-level control point wherever teams embed models, MCPs, or agent frameworks into production software. The practical signal is that AppSec, IAM, and supply-chain governance will need a shared view of provenance, privilege, and runtime reach before AI assets become impossible to unwind.

AI inventory debt: when organisations cannot trace which AI assets they ship, they accumulate governance debt that shows up first in audits, incident response, and access reviews. That debt is especially dangerous for AI-connected components because the boundary between code dependency and privileged runtime actor is already blurred. Practitioners should expect inventory, policy, and revocation to converge into one operational workflow.

Identity teams should prepare for more requests to govern non-human access paths created by AI components, especially where models and agents reach into data or operational tools. The actionable response is to make ownership, approval, and revocation explicit before the next wave of AI adoption spreads the control gap further.

For practitioners

• Define a single accountable owner for AI inventory Assign one function with authority over models, MCPs, agent frameworks, datasets, and AI libraries so other teams know where governance decisions live. Without clear ownership, inventory remains partial and no one can approve, reject, or retire AI assets consistently.
• Build a deterministic AI asset inventory Run discovery across repositories, container images, and services to identify embedded AI assets by type, version, provenance, and runtime location. Use that baseline to expose Shadow AI and to separate approved assets from unknown or unreviewed ones.
• Classify AI assets by access and trust level Tag each asset as model, dataset, prompt, library, MCP server, or agent framework, then map what data and tools it can access. This lets IAM and PAM teams decide where least privilege, approval, or segregation controls are required.
• Embed governance in CI/CD and code review gates Add policy checks for provenance, pinned versions, licensing, and prohibited AI dependencies before code merges or image publication. If a component cannot be verified, the pipeline should block it rather than relying on downstream review.
• Extend reviews to AI-beyond-code evidence Require AI-BOMs, dataset lineage, and runtime permission records alongside software inventories so audit teams can answer what is deployed and why. This closes the gap between discovery and accountability, especially when AI components cross team boundaries.

Key takeaways

• AI supply chain risk now includes models, MCPs, datasets, and agent frameworks that can behave like privileged non-human identities.
• The core failure is an AI inventory gap, where organisations cannot reliably answer what AI assets exist, where they run, or who owns them.
• Practitioners need deterministic discovery, provenance checks, and CI/CD policy gates before ungoverned AI assets become permanent dependencies.

Key terms

• AI Inventory: An AI inventory is a governed record of all AI-related assets, enriched with owner, purpose, access, and risk context. It turns discovery into something security, compliance, and IAM teams can use to make approval, review, and revocation decisions.
• MCP Server: An MCP server is a tool endpoint that connects an AI agent to external systems and data sources through Model Context Protocol. Because it extends what the agent can reach, it becomes part of the identity and access surface and must be reviewed like any other privileged connector.
• AI-BOM: An AI bill of materials is a structured inventory of the components that define an AI agent, including the model, prompt, tools, retrieval sources, and dependencies. In practice, it is the evidence base for review, change control, and risk assessment when the agent evolves after deployment.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.

What's in the full article

Checkmarx's full article covers the operational detail this post intentionally leaves for the source:

• The AI asset inventory workflow for models, datasets, agent frameworks, MCP servers, prompts, and libraries
• The specific governance actions the vendor recommends for building inventory and policy into delivery workflows
• The practical distinctions between AppSec scanning, supply-chain control, and AI-specific governance
• The article’s examples of how untracked AI assets create blind spots in real development environments

👉 The full Checkmarx article covers inventory gaps, governance blockers, and remediation actions in more detail.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps practitioners translate identity control principles into programmes that can govern fast-moving AI and non-human access paths.