Why do traditional AppSec tools miss AI supply chain risk?

Why This Matters for Security Teams

Traditional AppSec programs were designed to find flaws in application code, libraries, and deployment pipelines. AI supply chain risk sits one layer above and beside those familiar targets. The danger is not only malicious code, but also poisoned training data, tampered model artefacts, prompt injection paths, unsafe tool permissions, and connector trust relationships that reshape behaviour after deployment. That means an application can pass standard scanning and still expose sensitive data, take unintended actions, or produce unsafe outputs.

This is where current guidance suggests teams should treat AI components as governed assets rather than passive software dependencies. The NIST Cybersecurity Framework 2.0 remains useful for structuring governance, protection, detection, and response, but AI systems need additional controls for provenance, model integrity, and runtime oversight. In practice, the control gap appears when security reviews focus on source code and SBOMs while ignoring prompts, skills, model updates, and external tool access.

Many teams also underestimate how quickly a seemingly minor AI configuration change can alter the real security boundary. A new connector, a refreshed system prompt, or a modified retrieval source can expand access without any change to the application’s core code. In practice, many security teams encounter AI supply chain compromise only after an assistant has already leaked data or executed an unsafe action, rather than through intentional pre-deployment review.

How It Works in Practice

AI supply chain risk emerges from the full lifecycle of an AI-enabled system: data collection, model training or selection, packaging, deployment, orchestration, and ongoing updates. AppSec tools usually inspect artifacts that look like software, but they do not reliably assess the trustworthiness of model weights, prompt templates, agent tool definitions, retrieval sources, or policy layers. That is why a clean scan does not mean a safe AI system.

Security teams need to separate visible code risk from behavioural risk. A model may be internally safe enough, but its surrounding assets can create unsafe execution paths. For example, a retrieval source can introduce untrusted instructions, or an agent connector can grant access to files, tickets, or APIs that were never intended for autonomous use. NIST’s AI guidance, including the NIST IR 8596 Cyber AI Profile, pushes teams toward lifecycle controls that account for AI-specific threats instead of only classic code defects.

• Inventory every AI artefact, including models, prompts, skills, tools, embeddings, and retrieval corpora.
• Validate provenance for each artefact and record where it came from, who changed it, and when.
• Restrict tool permissions so agents only reach the minimum data and actions needed.
• Monitor inference-time behaviour for prompt injection, data exfiltration, and abnormal tool use.
• Apply change control to prompt and connector updates with the same discipline used for code releases.

This control model also intersects with Non-Human Identity governance when agent connectors, service accounts, API keys, and tokens are used to let AI systems act on behalf of users or workloads. That is where the OWASP Non-Human Identity Top 10 becomes directly relevant, because stolen or over-privileged machine identities can turn a model weakness into enterprise-wide abuse. These controls tend to break down in fast-moving MLOps environments where model, prompt, and connector changes are shipped independently and audit trails are incomplete.

Common Variations and Edge Cases

Tighter AI supply chain control often increases delivery overhead, requiring organisations to balance release speed against stronger provenance and approval checks. That tradeoff is real, especially when teams rely on third-party models, rapid experimentation, or dynamic agent tooling. Current guidance suggests there is no universal standard for how much review is enough; the right answer depends on the model’s exposure, data sensitivity, and whether the system can act autonomously.

Edge cases usually appear when AI components are sourced outside the normal software pipeline. A foundation model may be approved centrally, while the prompts and connectors are modified locally by a product team. A secure model can still become dangerous if a retrieval index is fed from unvetted documents or if a plugin can reach production systems through a shared service identity. That is why AI supply chain risk cannot be reduced to dependency scanning alone.

For some use cases, especially internal copilots with limited data access, the priority may be output validation and logging rather than deep model inspection. For high-impact or externally facing systems, stronger governance is warranted, including supplier review, model provenance checks, and tested rollback paths. Security teams should align these decisions with enterprise risk governance and the operational control expectations in the frameworks above, while acknowledging that best practice is still evolving for agentic systems.

Related resources from NHI Mgmt Group

• Why do traditional discovery tools miss shadow AI agents?
• Why does AI make software supply chain risk harder to control?
• Why do traditional security tools miss many AI security risks?
• Why do AI coding agents increase supply-chain risk?