Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

Shadow AI in the software supply chain: are AppSec controls keeping up?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 13010
Topic starter  

TL;DR: AI-generated code produces 1.7x more vulnerabilities than human-written code, while most organisations cannot inventory embedded models, MCP servers, agent frameworks, datasets, or prompts, according to Checkmarx. The governance problem is no longer just code volume but unseen AI components that create new supply chain attack paths and audit gaps.

NHIMG editorial — based on content published by Checkmarx: Shadow AI in the software supply chain and AI supply chain security

By the numbers:

Questions worth separating out

Q: How should security teams govern shadow AI in SaaS environments?

A: Security teams should inventory AI-enabled features, classify the data those features can touch, and enforce approved-use rules at the application and identity layers.

Q: Why does AI make software supply chain risk harder to control?

A: AI increases the amount of code produced, which reduces the time available for review and makes malicious or unauthorized changes harder to spot.

Q: What breaks when AI agents are not inventoried or classified?

A: When AI agents are not inventoried or classified, organisations lose the ability to assign risk, apply the right obligations, and prove control to auditors.

Practitioner guidance

  • Inventory AI artefacts as first-class assets Track models, agent frameworks, MCP servers, datasets, embeddings, and prompt files in the same governance process used for software dependencies.
  • Enforce deterministic discovery in CI/CD Use code and configuration signals to identify AI dependencies instead of relying on inference-based detection.
  • Gate unpinned models and unsafe agents Block floating model references, unverified weights, and autonomous tool use without explicit scope constraints.

What's in the full article

Checkmarx's full article covers the operational detail this post intentionally leaves for the source:

  • How the code-first discovery workflow identifies models, agent frameworks, MCP servers, datasets, and prompts from imports and configuration.
  • Which AI-specific risk checks apply to unverified weights, floating model references, unsafe autonomous agents, and exposed datasets.
  • How AI-BOM output fits into existing AppSec dashboards, CI/CD gates, and policy workflows without a separate platform.
  • Which compliance mappings support EU AI Act, ISO 42001, NIST AI RMF, and OWASP LLM Top 10 reporting.

👉 Read Checkmarx's analysis of shadow AI in the software supply chain →

Shadow AI in the software supply chain: are AppSec controls keeping up?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 12594
 

Shadow AI is becoming a supply chain governance problem, not just an AppSec visibility issue. Once models, prompts, MCP servers, and datasets are embedded in code, they function like operational dependencies with security implications. Traditional SCA and SBOM practices were not designed to capture those relationships, so the governance gap is structural rather than incidental. Practitioners should treat AI artefacts as governed assets from the moment they enter the build system.

A question worth separating out:

Q: How can organisations prove their AI controls are actually working?

A: Look for evidence that policy decisions are logged, sensitive prompts are being redacted or blocked when required, and approved AI interactions are traceable by identity and business context. Effective programmes produce audit-ready records, not just policy text. If the control cannot explain what happened in a session, it is not operational enough.

👉 Read our full editorial: Shadow AI in the software supply chain is breaking AppSec visibility



   
ReplyQuote
Share: