AI systems are assembled from parts you did not build: foundation models, fine-tuned weights, data sets, open source libraries, agent frameworks, plugins, MCP servers and third-party AI services. Each part can be compromised, and each often brings credentials and permissions with it. Recent attacks on AI packages, coding agent extensions and agent platforms show that the AI supply chain is now a primary target. This guide explains the components of the AI supply chain, the main threats, how an AI bill of materials (AI-BOM) helps, and the identity and integrity controls that matter most.
Key takeaways
- The AI supply chain includes models, data, code, tools and services, plus the credentials each component uses.
- An AI-BOM records what an AI system is made of, where each part came from and what it can access. It extends the software bill of materials (SBOM) to models, data sets and AI services.
- Verify integrity and provenance of models and packages, allowlist tools and MCP servers, and keep credentials away from unvetted components.
- Supply chain compromises in AI often aim at credentials. Least privilege and short-lived secrets limit the damage.
Components of the AI supply chain
| Component | Examples | Key risks |
|---|---|---|
| Foundation and fine-tuned models | Hosted model APIs; downloaded open-weight models | Backdoored or poisoned weights; unsafe serialisation formats that execute code on load |
| Data sets | Training, fine-tuning, evaluation and RAG sources | Poisoning; embedded secrets and personal data; licensing |
| Libraries and frameworks | AI SDKs, orchestration and agent frameworks, gateways | Malicious or compromised packages stealing credentials |
| Tools, plugins and MCP servers | IDE extensions, agent tools, community MCP servers | Tool poisoning; typosquatting; credential theft |
| Third-party AI services | SaaS AI features, AI assistants connected via OAuth | Vendor compromise exposing tokens and data |
| Prompts and agent configuration | System prompts, templates, agent instruction files, agent cards | Hidden instructions; tampering |
Recent AI supply chain incidents
- Amazon Q coding agent extension: data-wiping instructions injected into a published release.
- LiteLLM PyPI package: credentials stolen from users.
- Mastra npm packages: backdoors in AI-related packages.
- JetBrains Marketplace AI plugins: malicious plugins stealing AI API keys.
- CrewAI GitHub token leak: an internal token exposed through an AI platform.
- Salesloft Drift and Vercel via Context.ai: third-party AI tools whose tokens gave access to customer data.
What an AI-BOM should record
- Models: name, version, source, hash, licence, fine-tuning lineage, serialisation format.
- Data sets: sources, versions, collection dates, known sensitive content, licences.
- Software dependencies: libraries and frameworks with versions (the SBOM).
- Tools, plugins and MCP servers: source, version, permissions and credentials used.
- External AI services: providers, endpoints, data shared, authentication method.
- Identities and credentials: which NHIs the system uses and what they can access.
The CycloneDX standard supports machine learning BOMs, and SPDX 3.0 includes AI and data set profiles. OWASP's agentic guidance notes alignment with CycloneDX.
Controls
Integrity and provenance
- Download models only from approved sources; verify hashes and signatures where available.
- Prefer safe model formats that do not execute code on load; scan model files.
- Pin package versions, use lock files and route installs through a vetted internal registry.
- Sign internal models and artefacts and verify before deployment.
Allowlisting
- Maintain approved lists of models, AI libraries, IDE extensions, agent tools and MCP servers. See the MCP Security Guide.
- Review and pin third-party agent tools and detect changes to their definitions.
Credential containment
- Do not give unvetted components access to credentials; run them in sandboxes without ambient secrets.
- Use short-lived, least-privilege credentials so a compromised component gains little. See the Secrets Management Guide.
- Scope OAuth grants for AI vendors narrowly. See the SaaS and OAuth App Governance Guide.
Vendor assessment
- Assess AI vendors on data use, token protection, their own supply chain security and incident notification.
Monitoring and response
- Subscribe to advisories for AI packages and tools you use; be ready to rotate credentials quickly when one is compromised.
Practitioner checklist
- Build an AI-BOM for each production AI system, including models, data, dependencies, tools, services and credentials.
- Verify model and package integrity; use safe model formats.
- Route AI packages through a vetted registry and pin versions.
- Allowlist AI tools, plugins and MCP servers; detect changes.
- Keep credentials away from unvetted components; use short-lived, scoped secrets.
- Assess AI vendors and scope their access.
- Prepare to rotate credentials fast after an AI supply chain advisory.
Standards and references
- OWASP Top 10 for LLM Applications (2025): LLM03 Supply Chain, LLM04 Data and Model Poisoning
- OWASP Top 10 for Agentic Applications for 2026: ASI04
- CycloneDX Machine Learning BOM
- SPDX
- SLSA
- MITRE ATLAS
Related NHI Mgmt Group resources: AI Coding Agents Security Guide · CI/CD Pipeline Identity Security Guide · AI Infrastructure Workload Identity Guide · Agentic AI Security Guide
