AI agent supply chain security now depends on containment

At a glance

What this is: This analysis argues that AI agent supply chain security is failing because autonomous tooling now selects, installs, and executes dependencies without the human checkpoint traditional controls assumed.

Why it matters: It matters because IAM, PAM, and NHI teams must govern the real privileges and network reach of agents, not just the human account they impersonate.

👉 Read Xygeni's analysis of AI agent supply chain security and dependency risk

Context

AI agent supply chain security is becoming a governance problem because the control model was built around a human who read package names, reviewed diffs, and decided whether to proceed. In agentic workflows, that checkpoint disappears and dependency decisions happen at machine speed, often with direct access to tokens, registries, and build systems.

The result is not only software risk but identity risk: the agent inherits the developer’s access, which turns a convenience tool into a privileged runtime identity. That makes this topic relevant to IAM and NHI practitioners because the issue is who or what can act inside the pipeline, not just which package gets installed.

Key questions

Q: How should teams govern AI agents that can install dependencies?

A: Treat dependency installation as a privileged action that requires scoped authority, logging, and explicit approval for new packages. The agent should not inherit broad secrets or unrestricted registry access just because it can generate code. If the workflow must install, do it in a constrained environment with secret minimisation and dependency review before merge.

Q: Why does AI make software supply chain risk harder to control?

A: AI increases the amount of code produced, which reduces the time available for review and makes malicious or unauthorized changes harder to spot. It also introduces non-human actors into the development flow, so traditional assumptions about developer identity no longer hold. That combination expands the attack surface at the commit stage.

Q: What breaks when package review depends on human attention in agentic pipelines?

A: The review step stops being a reliable control because agents can generate large diffs, install code automatically, and continue working while no one is watching. In practice, the checkpoint becomes too slow and too shallow. Organisations should replace attention-based review with policies that prevent untrusted packages from reaching execution in the first place.

Q: Should organisations prioritise sandboxing or provenance for AI agent supply chains?

A: Sandboxing should come first because it limits what a compromised agent can reach right now. Provenance is still useful, but it records the build path after trust decisions have already been made. The safer sequence is containment first, then attestation, then tighter controls on package intake and prompt-driven dependency discovery.

Technical breakdown

Hallucinated package names become a repeatable attack surface

AI models that recommend libraries can invent package names, and attackers can register those names before a developer or agent resolves them. The important technical point is predictability: if the same hallucinated names recur across prompts, the attacker can industrialise the trap. In some cases the malicious package is only the delivery vehicle; in others, the malicious content sits in README text, skill files, or MCP descriptions that an agent consumes as trusted context. Traditional code scanning misses this because the payload may be plain text rather than executable malware.

Practical implication: treat package discovery and dependency resolution as a control boundary, not a convenience step.

Agent permissions collapse source, build, and identity trust

When an agent runs as the developer, it inherits the developer’s environment variables, cloud tokens, registry credentials, SSH keys, and local network reach. That collapses three separate questions into one: what code is being added, what build action is being taken, and whose identity is authorising it. If the agent can trigger installs or pipeline steps autonomously, then a malicious dependency can execute with the same blast radius as the human account it impersonates. The trust failure is architectural, not just behavioural.

Practical implication: separate agent execution identities from human developer identities and reduce inherited privilege aggressively.

Provenance proves the build path, not the trustworthiness of the input

SBOMs and build provenance tell you what happened during a build, and that is useful. They do not prove that the package, commit, or artefact entering the pipeline deserved trust in the first place. If an attacker poisons the input before the artefact exists, the resulting attestation can be perfectly valid and still describe a malicious outcome. That is why dependency vetting, cache integrity, and pre-adoption delay windows remain necessary controls alongside attestation.

Practical implication: pair provenance with source trust controls, cache protection, and delayed adoption for new packages.

Threat narrative

Attacker objective: The attacker aims to turn an AI agent's delegated build authority into code execution, secret theft, or supply chain compromise inside the development pipeline.

1. Entry occurs when an attacker seeds a hallucinated package name, malicious README content, or a compromised dependency into the agent's search path.
2. Escalation happens when the agent installs the package or follows the malicious instruction using the developer's inherited credentials and build permissions.
3. Impact follows when the compromised agent executes code, poisons the build, or exposes environment secrets through the pipeline.

NHI Mgmt Group analysis

AI agent supply chain security is really an identity problem disguised as AppSec. The article makes clear that the agent inherits the developer's access and acts faster than any human review cycle can operate. That means the real control question is who or what is authorised to install, execute, and publish within the pipeline. Practitioners should treat agent identity, not just package hygiene, as the primary governance boundary.

Hallucinated dependency names create a new form of trust collapse that attackers can scale. The combination of model-generated package suggestions, repeatable fake names, and plain-text prompt injection turns dependency discovery into an attacker-controlled input stream. This is not a one-off typo problem. It is a predictable naming surface that can be farmed across environments. The named concept here is dependency hallucination risk, and it belongs in every agentic software governance model.

Provenance is necessary but insufficient when the build is poisoned before attestation begins. SBOMs and signed provenance can prove that a build followed the expected path, but they cannot rescue an untrusted input that was accepted upstream. That exposes a governance assumption many teams still make, namely that build attestations validate trust rather than record process. Practitioners should stop treating attestation as a substitute for source validation.

Sandboxing is now a first-class containment control for software supply chains. The article is right to frame isolation, cooldown windows, and egress limits as practical defenses rather than advanced options. Once an agent can reach package registries, cloud tokens, and local credentials, the security model depends on containment, not policy intent. Teams that cannot isolate agents are effectively letting a delegated identity browse the open internet from inside the build.

The control stack must shift from human review to machine-constrained execution. Human approval steps degrade quickly when agents generate thousands of lines or act continuously. Security teams need controls that remain effective when no one is reading the output in real time. That means least privilege, isolated execution, network restriction, and monitored package intake should be treated as baseline operational requirements.

What this signals

Dependency hallucination risk will become a standard governance term as more pipelines let models suggest, select, and install software without human intervention. For identity and security teams, that means the control conversation has to shift from review quality to execution constraints, including isolated runtime, egress control, and prompt-to-install approval boundaries. The relevant external control model is OWASP Agentic AI Top 10, especially where tool use and identity abuse overlap.

Machine-constrained execution is now the practical boundary for secure agentic delivery. Once an agent can browse registries, fetch dependencies, and trigger installs on its own, policy documents stop being operational controls. Teams should inventory every token, mount, and network path the agent can touch, then reduce that set until a compromised agent cannot reach the rest of the environment. That is where NIST AI Risk Management Framework governance becomes concrete rather than aspirational.

For practitioners

• Isolate agent execution from developer credentials Run coding and build agents in a microVM or container with only the current project mounted, and do not expose host tokens, SSH keys, or cloud credentials to the agent runtime.
• Add a cooldown window before adopting new dependencies Delay installation of newly published packages for several days so active supply chain attacks have time to surface in advisories, scans, or community reporting before they reach the build.
• Separate dependency discovery from dependency installation Require a human or a tightly constrained policy engine to approve any package name the agent proposes, especially when the name came from model output rather than a curated allow list.
• Block outbound paths the agent does not need Restrict egress so build agents can reach only approved registries, artifact stores, and internal services, then log any attempt to connect beyond those boundaries.
• Scan the container image, not just the source tree Add malware and CVE visibility to the build pipeline so base-image vulnerabilities and injected artefacts are surfaced before merge, not after deployment.

Key takeaways

• AI agent supply chain risk is no longer about careless developers alone, because autonomous tooling now controls the dependency decision itself.
• Hallucinated package names, plain-text malicious instructions, and inherited credentials combine into a repeatable attack path that scanners often miss.
• Containment controls such as sandboxing, egress restriction, and dependency cooldowns matter more than review-based policies when no human is in the loop.

Key terms

• Dependency Hallucination: Dependency hallucination happens when a generative model invents a package, library, or module name that sounds real but does not exist. Attackers can register those names on public registries and turn the mistake into a supply chain attack, making verification of generated dependencies a critical control.
• HalluSquatting: HalluSquatting is a supply chain attack in which an attacker plants malicious instructions in plain text, such as a README, skill file, or MCP description, and waits for an agent to hallucinate the same tool or repository name. The agent then follows the malicious context as if it were legitimate.
• Agent execution identity: An agent execution identity is the identity assigned to the runtime that performs actions on behalf of a user or application. It is not the same as the human user or the model, and it must be scoped, logged, and revocable as its own non-human identity.
• Build provenance: Build provenance is the evidence chain showing where a software artefact came from, how it was assembled, and which identities and keys were used. It is essential when teams need to prove that an embedded release was produced from trusted source and controlled inputs.

What's in the full article

Xygeni's full blog post covers the operational detail this post intentionally leaves for the source:

• Step-by-step containment patterns for agent runtimes, including microVM and container isolation choices
• Pipeline hardening detail for dependency intake, install approval, and artifact scanning before merge
• Operational guidance on cooldown windows, egress control, and package trust checks for agentic builds
• Incident examples and exploit chain details that show how these attacks move from hallucination to execution

👉 Xygeni's full post covers the attack chain, containment controls, and pipeline hardening details

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps practitioners build control models that still work when software acts on its own.