AI BOMs expose the governance gap in appsec supply chains

At a glance

What this is: This is an AppSec analysis of why SBOMs do not adequately document AI components and why AI-BOMs are emerging as the missing inventory artifact.

Why it matters: It matters because security, compliance, and identity teams need evidence of what AI components exist, who approved them, and what access or governance controls apply across production software.

By the numbers:

• 43% have no formal governance over which AI components their developers are permitted to use.
• 70% expect AI components in production by the end of 2026, with 24% already there today.
• 27 days

👉 Read Checkmarx's analysis of AI-BOM governance for AppSec supply chains

Context

AI-BOMs matter because AI components are not inventory-managed the same way as packages. A model call in source code, an MCP server in configuration, or an agent framework in the dependency graph can influence runtime behaviour without appearing as a discrete, governable asset in the way AppSec teams are used to tracking.

The identity and governance problem is not limited to software supply chain documentation. When AI components can invoke tools, consume datasets, or operate through external services, teams need evidence of approval, scope, provenance, and lifecycle control. That is why the article’s starting position is typical for modern AppSec programmes: the tooling gap appears only once AI reaches production, not during experimentation.

Key questions

Q: What breaks when organisations rely on SBOMs alone for AI-enabled applications?

A: SBOMs can show package dependencies, but they usually miss models, agents, prompts, MCP servers, and dataset provenance. That leaves teams unable to produce audit-ready evidence for how AI behaves in production. The practical failure is governance blind spots, not just inventory gaps, because reviewers cannot tell what AI exists, who approved it, or what authority it has.

Q: Why do AI components create a documentation problem for AppSec and IAM teams?

A: AI components blur the line between software inventory and runtime authority. A model reference, agent tool, or MCP connection can change application behaviour without appearing as a discrete package, so security teams lose the ability to link identity, approval, and provenance to a stable artefact. That makes governance and accountability harder to prove.

Q: How do teams know whether AI-BOM output is actually useful for compliance?

A: Useful AI-BOM output is reproducible, traceable, and release-specific. Every entry should map back to a file, line number, or configuration object, and the inventory should be regenerated whenever the application changes. If the evidence cannot be recreated from source artefacts, it is too weak for audit or regulatory review.

Q: Who should own accountability for undocumented AI components in production?

A: Accountability should sit with the application owner, security, and the governance function that approves software change, not with auditors after the fact. For regulated environments, the organisation must be able to show who approved the component, what it can do, and how its scope is controlled across its lifecycle.

Technical breakdown

Why SBOMs miss AI components in production software

A software bill of materials is designed to enumerate discrete packages declared through manifests and dependency graphs. AI components often arrive through API calls, configuration strings, runtime services, and embedded prompts, so they can be operationally real without being inventory-visible. That matters because an SBOM can show the SDK, but not the model version, provenance, tool scope, or dataset licensing that define risk. AI-BOMs extend the inventory model to cover those artefacts, making governance evidence machine-readable and audit-friendly.

Practical implication: treat SBOM output as incomplete for AI-enabled applications and add AI-specific inventory rules at build time.

Deterministic detection versus model-based guessing

The article argues for deterministic detection because audit evidence must be traceable to source code or configuration. In practice, this means identifying AI components through imports, API call patterns, configuration files, and string constants rather than confidence scoring. Deterministic outputs support repeatability, change control, and auditor review, while model-based classification can produce ambiguous findings that are hard to defend in regulated environments. For AppSec, the key issue is evidentiary integrity, not just detection coverage.

Practical implication: require file-level and line-level traceability for every AI component finding before it enters governance reporting.

AI-BOMs and the governance trail for agents, prompts, and MCP servers

AI-BOM scope needs to include more than model names. Agents can invoke tools, prompts shape behaviour, MCP servers expose external connections, and datasets can create licensing or provenance obligations. These are governance objects as much as technical components, because they influence what the system can do and under what authority. In identity terms, the question becomes who approved the component, what it can access, and how its scope changes over time. That makes AI-BOMs a bridge between AppSec evidence and identity governance.

Practical implication: map AI component inventory to approval, access scope, and lifecycle ownership before release.

NHI Mgmt Group analysis

AI-BOMs are becoming the governance boundary between code inventory and AI accountability. Traditional SBOM practice assumes the security-relevant unit is a versioned package. That assumption fails when models, agents, prompts, and MCP servers affect runtime behaviour without appearing as first-class dependencies. The operational consequence is that audit evidence, risk review, and component ownership now depend on a richer inventory model. Practitioners should treat AI-BOMs as the control point that links technical detection to governance evidence.

AI governance debt: organisations are accumulating undocumented AI usage faster than they can build controls around it. The article’s 43% governance gap shows that many teams already lack formal boundaries on developer-chosen AI components. That creates a familiar identity problem in a new form: uncontrolled component adoption precedes policy, review, and evidence. The result is not just compliance friction but an inability to explain what is running in production. Practitioners should assume documentation lag until AI component intake is controlled earlier in the lifecycle.

Identity and access controls now extend to the AI supply chain. An AI component is not merely a model reference. It can embody tool access, dataset dependence, and update authority that alter the security boundary of the application. That means identity governance must cover approval rights, provenance checks, and scope review for non-human components as well as human actors. Practitioners should align AI inventory with who can introduce, modify, or retire those components.

Regulatory alignment is shifting from artifact collection to evidence quality. CRA, EU AI Act, NIST AI RMF, and ISO 42001 all create pressure for traceable documentation, but the differentiator is whether the evidence is operationally useful. A machine-readable AI-BOM only matters if it can be regenerated per release and mapped back to source. That pushes AppSec toward continuous evidence generation rather than after-the-fact reporting. Practitioners should build for release-level traceability, not audit-day reconstruction.

What this signals

AI-BOM is the beginning of evidence-led AI governance, not a documentation exercise. AppSec programmes that treat AI inventory as a one-time catalogue will fall behind the rate of change in production software. The stronger model is continuous evidence generation, where build pipelines produce traceable records that compliance, legal, and security can all consume. That is especially important where AI components can be introduced through configuration as easily as code.

AI governance debt will surface first as identity and approval ambiguity. Once teams cannot answer who approved a model, agent, or MCP connection, the control problem has already moved upstream of scanning. The right next step is to connect AI component inventory to change management, approval workflows, and release gating. For broader standards alignment, teams should anchor evidence against the NIST Cybersecurity Framework 2.0 and, where access control and authenticator management are in scope, NIST SP 800-53 Rev 5 Security and Privacy Controls.

For practitioners

• Define an AI component inventory policy Classify what must be tracked, including models, agents, prompts, MCP servers, datasets, SDKs, and external service references. Make the policy explicit enough that engineering teams can apply it consistently during intake and release.
• Require deterministic evidence for AI detections Use source-linked findings from imports, configuration files, API calls, and string constants so every AI component can be traced to a file and line number. Reject inventory entries that cannot be reproduced from code or build artefacts.
• Embed AI-BOM generation into CI/CD Generate AI-BOM output at build or release time and version it alongside the software artefact. That keeps evidence current, reduces audit-day reconstruction, and makes review part of the normal delivery pipeline.
• Map AI component scope to approval rights Record who approved each AI component, what tools it can invoke, and which datasets or providers it depends on. Treat scope expansion as a governance event that needs review before deployment.

Key takeaways

• Traditional SBOMs do not capture the AI components that now influence application behaviour, which leaves governance and audit evidence incomplete.
• AI-BOM adoption is being driven by real compliance pressure, but its value depends on deterministic, source-linked detection rather than probabilistic guesses.
• Teams that connect AI inventory to approval rights, release workflows, and identity governance will have a defensible control model when audits arrive.

Key terms

• AI-BOM: An AI bill of materials is a structured inventory of the components that define an AI agent, including the model, prompt, tools, retrieval sources, and dependencies. In practice, it is the evidence base for review, change control, and risk assessment when the agent evolves after deployment.
• Deterministic Detection: Deterministic detection identifies a component from concrete source artefacts such as imports, configuration, API calls, or string constants. It is valuable in audit-heavy environments because it produces repeatable findings that can be traced back to specific code locations and reviewed without relying on model scores or inference guesses.
• MCP Server: An MCP server is a tool endpoint that connects an AI agent to external systems and data sources through Model Context Protocol. Because it extends what the agent can reach, it becomes part of the identity and access surface and must be reviewed like any other privileged connector.
• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.

What's in the full article

Checkmarx's full article covers the operational detail this post intentionally leaves for the source:

• How Checkmarx generates AI-BOM output in CycloneDX format across release pipelines.
• How deterministic detection is implemented using imports, API calls, configuration files, and string constants.
• How policy enforcement works for provider allowlists, blocklists, and floating model references in CI/CD.
• How the output is mapped to CRA, EU AI Act, NIST AI RMF, and ISO 42001 reporting needs.

👉 The full Checkmarx article covers deterministic detection, CycloneDX output, and compliance mapping details.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for practitioners building governance across IAM, secrets, machine identity, and AI-driven access paths.