OWASP LLM Top 10 LLM03 Supply Chain Guide: Securing Models, Datasets and Plugins

An LLM application inherits the weaknesses of every model, adapter, dataset, library, conversion service and hosting platform it is built from, and most teams cannot list those components, let alone verify them. LLM03:2025 Supply Chain is the OWASP Top 10 for LLM Applications entry for that problem. This guide turns its nine risk examples and ten mitigations into an assessable requirement set, with the evidence an assessor should ask for, the identities that protect each link in the chain, and a phased rollout. Model file formats and safe loading are covered in depth in the Model Supply Chain Guide and AI-BOM content in the AI Supply Chain and AI-BOM Guide; this page links to them rather than repeating them. It is written for AppSec, ML platform and third-party risk teams.

Key takeaways

• LLM03 is wider than software dependencies. OWASP adds pre-trained models, LoRA adapters, datasets, collaborative merge and conversion services, on-device models and supplier terms of service to the usual package risks.
• Model provenance is weak by default. OWASP states there are currently "no strong provenance assurances in published models"; hashes, signatures and pinning have to compensate.
• Benchmarks are not assurance. PoisonGPT's edited model scored within 0.1% of the original on a toxicity benchmark, which is why OWASP asks for red teaming in your own use case.
• Build and publishing identities are the soft spot. Bot tokens, CI publishing tokens and hub write tokens were the route in for the Safetensors conversion research and the LiteLLM PyPI compromise.
• Terms of service are a supply chain risk. OWASP lists unclear T&Cs and privacy policies, including a supplier switching to opt-out training on your data, alongside technical tampering.

What it is and who it applies to

The official entry is LLM03:2025 Supply Chain on genai.owasp.org (page last modified May 2025). The plan title of this page matches the official risk name. OWASP says LLM supply chains "are susceptible to various vulnerabilities, which can affect the integrity of training data, models, and deployment platforms", and that in machine learning the risk "also extend[s] to third-party pre-trained models and data". Poisoning overlaps with LLM04 Data and Model Poisoning; OWASP says LLM03 focuses on the supply chain aspect. Plugins and tools, named in this page's title, sit in this entry as externally supplied code; their run-time permissions belong to LLM06. See the OWASP LLM Top 10 LLM06 Excessive Agency Guide.

Who it applies to: anyone who fine-tunes or self-hosts open-weight models, loads models, adapters or datasets from a public hub, consumes a hosted model API, ships a model on device, or builds LLM features on open-source frameworks. Organisations that only consume a hosted API still own risk examples 3 (outdated or deprecated models) and 9 (terms and privacy), plus the package risks in their own application code.

Structure and key requirements

The entry has nine common examples of risk, ten prevention and mitigation strategies and thirteen sample attack scenarios, and maps to MITRE ATLAS AML.T0010 ML Supply Chain Compromise (named AI Supply Chain Compromise in current ATLAS data). The table pairs each risk with the OWASP strategy that addresses it and the evidence to request.

Risks the entry adds beyond classic software supply chain

• LoRA adapters at serving time: OWASP names vLLM and OpenLLM support for downloading and applying adapters to a deployed model. vLLM's own documentation says runtime adapter loading (enabled with VLLM_ALLOW_RUNTIME_LORA_UPDATING and the /v1/load_lora_adapter endpoint) "comes with security risks" and should not be used in production unless the environment is isolated and fully trusted, and that its Hugging Face Hub resolver plugin is "insecure and not intended for use in production". Load adapters at start-up with --lora-modules from an approved path instead.
• Conversion and merge services: HiddenLayer showed that Hugging Face's Safetensors conversion Space loaded submitted PyTorch files with torch.load(), so a crafted model could run code in the service container, where the SFconvertbot token was available as an environment variable. That token could open pull requests on any repository under the bot's trusted name, and HiddenLayer found organisations hosting 905 models had accepted pull requests from the bot.
• Edited and look-alike models: Mithril Security's PoisonGPT edited GPT-J-6B with ROME to change one factual association, uploaded it under a look-alike organisation name missing one letter, and measured only a 0.1% accuracy difference on the ToxiGen benchmark.
• Hardware and hosting layer: LeftoverLocals (CVE-2023-4969), cited by OWASP, let one process recover data left in GPU local memory by another; Trail of Bits measured about 181 MB per LLM query on an AMD Radeon RX 7900 XT running a 7B model with llama.cpp, enough to reconstruct the response. NVIDIA and Arm said they were not affected. The practical control is avoiding untrusted multi-tenant GPU sharing for sensitive inference. See the Confidential AI and GPU Enclaves Guide.
• AI framework infrastructure: the ShadowRay scenario concerns CVE-2023-48022, which NVD describes as remote code execution via the Ray job submission API and marks as disputed, with the vendor stating Ray is not intended for use outside a strictly controlled network and offering token authentication from version 2.52.0. Treat AI orchestration dashboards and job APIs as privileged management planes, never internet facing.

Identity and NHI requirements

Every link in an LLM supply chain is guarded by a non-human identity: hub tokens that publish models, CI tokens that publish packages, bot tokens used by conversion services, registry credentials for internal mirrors and API keys for hosted models. Compromise of one of these is how tampered components reach consumers.

• CI secrets reach every tool in the job: in the LiteLLM PyPI package breach a tampered security scanner in the CI pipeline read the PYPI_PUBLISH token, and the attackers used it to upload two malicious LiteLLM versions that harvested cloud credentials and LLM API keys from installing hosts.
• Platform secrets are part of the chain: the Hugging Face Spaces breach exposed a subset of stored Spaces secrets; Hugging Face revoked affected tokens and made fine-grained tokens the default. See the Hugging Face Security Guide.
• Development hosts hold the keys: the PyTorch nightly torchtriton compromise (25 to 30 December 2022) installed a binary that collected environment variables, $HOME/.ssh/* and .gitconfig from Linux hosts that installed the nightly via pip, because pip preferred a same-named PyPI package over PyTorch's own index. Fine-tuning workstations need the same dependency controls as production. See the OWASP NHI Top 10 NHI3 Vulnerable Third-Party NHI Guide.

Implementation steps

1. Inventory components: list every model, adapter, dataset, embedding model, framework, serving engine and hosted API each LLM feature uses, with source, version or commit, digest, licence and owner. Use CycloneDX, whose ML-BOM capability covers models, datasets and configurations, as OWASP suggests.
2. Set intake policy: approved sources only; non-executable weight formats preferred; models pinned to a commit and digest; adapters subject to the same rules as base models. See the Model Supply Chain Guide.
3. Stand up an internal registry: mirror approved models, adapters and packages; let only the intake pipeline identity write to it; point all training and serving environments at the mirror.
4. Verify integrity: record SHA-256 digests at intake and verify at load. Where publishers sign, verify signatures; the OpenSSF Model Signing specification signs a manifest of every file's hash in a detached Sigstore bundle, and supports bare keys, private PKI and Sigstore. Sign your own models the same way.
5. Evaluate before adoption: red team each candidate model in the intended use case (OWASP strategy 3) and keep the report tied to the digest; repeat after any update. See the AI Red Teaming Guide.
6. Harden serving: disable runtime adapter loading and remote model downloads in production; load from the registry at start-up; keep orchestration APIs on private networks with authentication.
7. Lock down publishing identities: move to trusted publishing, scope CI secrets to the job that needs them, pin CI tools by digest, and use fine-grained hub tokens.
8. Review suppliers and terms: record training opt-out, retention and change-notification terms for each hosted model; re-review on every terms change.
9. Roll out in audit mode: run the intake checks as warnings for one or two release cycles, fix the backlog of unpinned and unsigned components, then enforce for a pilot team before blocking organisation-wide. Keep the previous approved digest so a rejected update can be rolled back immediately.

Decision guidance

Evidence and assessment

• Inventory evidence: a current BOM per LLM feature listing models, adapters, datasets, packages and hosted services with digests and licences.
• Integrity evidence: intake logs showing digest and signature checks, and load-time verification in serving.
• Assurance evidence: red-team and evaluation reports per model digest; dependency scan results for training, fine-tuning and serving environments.
• Identity evidence: list of publishing, hub, registry and API identities with scopes, owners and last rotation; proof of trusted publishing.
• Supplier evidence: reviewed T&Cs and privacy terms with dates, and deprecation dates for each hosted model.

What to monitor: loads of models or adapters not in the registry, digest mismatches at load, new outbound downloads from model hubs in production, changes to pinned commits, publishing events outside the release workflow, and supplier terms or deprecation notices.

Mapping to other frameworks

See the NIST SP 800-161 Cyber Supply Chain Risk Management Guide and the SLSA and Build Provenance Guide for the general supply chain programme.

Common pitfalls

• Trusting the name: loading a model by organisation and repository name without a pinned commit; look-alike names, as in PoisonGPT and OWASP's WizardLM scenario, and later changes both slip through.
• Trusting the benchmark: accepting a leaderboard score as evidence of safety, when OWASP notes models can be fine-tuned to pass published benchmarks.
• Scanning the app but not the training box: fine-tuning and notebook environments hold data and credentials but are often outside dependency controls, as the torchtriton nightly compromise showed.
• Runtime adapter loading in production: enabling convenient dynamic LoRA loading that vLLM's own documentation says is for isolated, fully trusted environments only.
• Merging bot pull requests unread: treating conversion or merge bot changes as trusted because of the bot's identity.
• Stale publishing rights and tokens: long-lived tokens in CI can be read by any tool in the job, and dormant accounts keep their rights; in the Mastra npm supply chain attack a former contributor's account that still had publish rights was used to backdoor more than 140 AI framework packages, after which Mastra disabled token-based publishing.

Practitioner checklist

• Maintain a CycloneDX BOM per LLM feature covering models, adapters, datasets, packages and hosted services.
• Pin every model and adapter to a commit and SHA-256 digest and verify at load.
• Serve only from an internal registry that only the intake pipeline can write to.
• Prefer non-executable weight formats and verify publisher signatures where offered; sign your own models.
• Red team each model in its intended use case and tie the report to the deployed digest.
• Disable runtime adapter loading and remote hub downloads in production serving.
• Scan dependencies in training, fine-tuning and serving environments, not only the application.
• Replace long-lived publishing tokens with trusted publishing and scope CI secrets per job.
• Record licence, training opt-out, retention and deprecation terms for every model supplier and review on change.
• Keep AI orchestration and job APIs on private networks with authentication.

Standards and references

• OWASP: LLM03:2025 Supply Chain
• OWASP: Top 10 for LLM Applications 2025
• MITRE ATLAS: AI Supply Chain Compromise
• NIST: SP 800-53 Rev. 5
• OpenSSF: Model Signing Specification
• CycloneDX: Machine Learning Bill of Materials (ML-BOM)
• vLLM: LoRA Adapters
• HiddenLayer: Silent Sabotage, Hijacking Safetensors Conversion on Hugging Face

Related NHI Mgmt Group resources: OWASP Top 10 for LLM Applications Guide · Model Supply Chain Guide · AI Supply Chain and AI-BOM Guide · Self-Hosted LLM Security Guide · Software Supply Chain Security Guide