AI workloads expose the gaps in HPC security overlays

At a glance

What this is: This analysis argues that AI workloads on high-performance computing systems invalidate core assumptions in NIST SP 800-234, especially around perimeter scanning, job isolation, and integrity verification.

Why it matters: It matters because identity, supply-chain, and runtime controls must now govern AI workloads that behave more like living systems than deterministic jobs, with direct implications for NHI, workload identity, and privileged automation.

By the numbers:

• Research across 41 studies found attack success rates exceeding 60 percent from manipulation of 100 to 500 training samples.
• Systems with least-privileged AI access had a 17% incident rate vs 76% for over-privileged systems, according to the 2026 Infrastructure Identity Survey.

👉 Read SentinelOne's analysis of AI workloads, HPC security gaps, and supply chain attacks

Context

AI workloads on HPC are not just larger jobs, they are different jobs from a security perspective. Deterministic overlays assume predictable execution, bounded inputs, and clear inspection points, but AI training and inference introduce mutable models, dynamic behaviour, and supply-chain dependencies that do not fit those assumptions.

The primary governance problem is that traditional HPC controls were built around code and data that behave consistently from one run to the next. Once the workload can learn, mutate, or be updated through automated pipelines, the question shifts from perimeter protection to whether the security model can still describe what is trusted, when, and by whom.

For identity teams, that makes AI workloads an NHI and workload identity problem as much as an infrastructure one. Service accounts, tokens, signing flows, and delegated automation now sit in the same control plane as training data, model artifacts, and runtime telemetry.

Key questions

Q: How should security teams govern AI workloads running on HPC systems?

A: Security teams should govern AI workloads as continuously changing execution environments, not as static batch jobs. That means combining workload identity, behavioural monitoring, provenance checks, and runtime containment so trust is not assumed to persist after job start. The control model must cover the full execution window, not just launch-time authorisation.

Q: Why do AI workloads break traditional HPC security assumptions?

A: AI workloads break traditional HPC assumptions because they can change behaviour during execution, rely on mutable model artifacts, and consume trusted supply-chain inputs that remain legitimate while carrying malicious intent. Deterministic overlays were built for repeatable workloads, so they cannot fully account for runtime drift, poisoned data, or post-launch manipulation.

Q: What do security teams get wrong about AI supply chain risk?

A: Many teams focus on signed code and forget that AI systems also depend on training data, checkpoints, and model weights. A package can be validly signed and still deliver a harmful payload, while a model can be clean at deployment and later be altered or poisoned through upstream artifacts. Provenance must follow the model, not just the software.

Q: Who is accountable when AI workloads introduce corrupted outputs or unsafe actions?

A: Accountability sits with the teams that approve the workload, own the data and model lineage, and control the credentials that let automation publish, deploy, or update artifacts. In practice, that means platform owners, IAM teams, and model governance leads share responsibility for the trust chain, not just the security team.

Technical breakdown

Why deterministic HPC controls fail for AI workloads

NIST SP 800-234 is tuned for workloads that are repeatable, inspectable, and relatively stable across runs. That works for simulation jobs where the same inputs produce the same outputs, but AI training and inference can change behaviour as models adapt, weights update, or pipelines swap components. Controls such as perimeter scanning, memory clearing between jobs, and load-time attestation assume trust can be validated at discrete checkpoints. AI breaks that model because the meaningful security event may happen after execution starts, not before it begins.

Practical implication: Treat AI execution as a continuous monitoring problem, not a pre-flight compliance check.

Why the AI supply chain extends beyond signed software

The article shows three separate trusted-channel attacks: a poisoned Python package, a legacy token abuse in npm, and a signed binary distributed from a legitimate vendor domain. That pattern matters because AI introduces an additional layer of supply chain risk in training data, checkpoints, and model weights. Conventional software supply-chain controls can tell you whether a package was signed, but they do not tell you whether the data used to train or tune a model was trustworthy, complete, or altered in transit.

Practical implication: Extend supply-chain controls from code provenance to model and data provenance.

How runtime telemetry becomes the control that matters

The article’s core technical point is that behavioural monitoring can catch what signatures miss. In AI environments, useful telemetry includes CPU and GPU behaviour, memory access patterns, inter-node communication, output distributions, and anomalous process chains. The goal is not to identify every possible malicious payload in advance. It is to define what legitimate training or inference should look like, then stop execution when the workload departs from that baseline.

Practical implication: Build detection around workload behaviour, not just static indicators of compromise.

Threat narrative

Attacker objective: The attacker wants to insert malicious code, data, or model behaviour into trusted AI and HPC workflows without triggering signature-based detection or human review.

1. Entry occurs through trusted delivery channels, including compromised package repositories, legacy access tokens, and vendor distribution infrastructure.
2. Escalation happens when malicious payloads run through automated pipelines, unrestricted AI agents, or signed binaries that preserve legitimacy while subverting intent.
3. Impact follows when poisoned training, modified inference, or compromised runtime behaviour produces false outputs, unsafe actions, or silently corrupted model state.

Breaches seen in the wild

• LiteLLM PyPI package breach — LiteLLM PyPI supply chain attack, credentials stolen from users.
• Shai Hulud npm malware campaign — Shai Hulud campaign: npm malware exposed secrets on GitHub.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI workloads expose an identity governance gap, not just an HPC tuning problem. The issue is not that existing controls are weak in general, but that they were designed for predictable execution rather than adaptive systems. Once a workload can accept updates, invoke automation, or change behaviour at runtime, the governance model has to account for machine identity, delegated access, and continuous trust. Practitioners should treat AI workload governance as a cross-domain identity problem rather than a pure infrastructure issue.

Runtime trust must replace checkpoint trust when model behaviour can change after launch. The article shows why load-time integrity checks and perimeter scans do not close the gap for long-running AI systems. This aligns with OWASP-NHI and NIST CSF thinking: the control objective is not merely to authenticate the workload, but to keep validating its behaviour as conditions change. Security teams should expect the trust boundary to move from job start to the entire execution window.

The AI supply chain now includes artifacts that traditional software bills of materials do not describe. Signed packages, valid tokens, and legitimate vendor channels are no longer sufficient evidence of trust when the real payload may live in the model, training set, or checkpoint chain. Model provenance blind spot: this is the specific failure mode the article exposes, because the current control model tracks software lineage better than it tracks AI lineage. Practitioners need to understand that unverified model artifacts create a governance problem, not just an operational one.

Least privilege for AI systems only works when access scope is tied to actual runtime intent. The article’s examples show how unrestricted automation and stale credentials can move harmful code through approved channels without a human checkpoint. That is the same structural issue seen in over-privileged NHI estates, where authorised access outlives the business need that justified it. The implication is that access governance must now be able to express and enforce the narrowest possible runtime purpose.

AI security will increasingly be judged by whether organisations can prove controlled execution, not just known-good inputs. The field is moving toward behavioural assurance for training and inference, because that is where the meaningful attack surface lives. For practitioners, the benchmark is no longer whether a job was allowed to start, but whether the environment can demonstrate continuous confidence in what the workload did after it started. That is the standard emerging for AI governance.

From our research:

• Systems with least-privileged AI access had a 17% incident rate vs 76% for over-privileged systems, according to the 2026 Infrastructure Identity Survey.
• Only 44% of organisations have implemented any policies to manage their AI agents, despite 92% agreeing that governing AI agents is critical to enterprise security.
• That gap is echoed in Top 10 NHI Issues, which helps teams translate AI access sprawl into a broader identity governance programme.

What this signals

Identity teams should expect AI workload governance to converge with NHI governance faster than infrastructure teams anticipate. Once automation can publish packages, update pipelines, or run inference without human approval, the access problem looks less like a one-off workload exception and more like an ongoing identity lifecycle issue. The control question shifts to who owns the credential, who reviews its scope, and who can revoke it when behaviour changes.

Least privilege is becoming the dividing line between controllable AI and operational drift. With 70% of organisations already granting AI systems more access than human employees, per the 2026 Infrastructure Identity Survey, the governance gap is structural rather than incidental. Teams that keep treating AI access as a temporary exception will struggle to contain blast radius when model-serving or training paths are abused.

Model provenance blind spot: this is the term security leaders should start using when discussing AI supply chain assurance. It captures the gap between knowing a package is signed and knowing whether the model, checkpoint, or training data beneath it is trustworthy. That distinction will increasingly shape how organisations align AI security work with NIST Cybersecurity Framework 2.0 and identity governance.

For practitioners

• Separate AI security monitoring from the workload itself Reserve dedicated CPU resources for behavioral monitoring so the protection layer cannot be starved, delayed, or filtered by the workload it watches. Treat the monitor as infrastructure, not a co-scheduled helper, and place it outside the execution path that the AI job can influence.
• Extend provenance checks from software to model artifacts Track training data, intermediate checkpoints, and deployed weights as governed artifacts with clear ownership and integrity controls. If a model enters the environment without verifiable lineage, treat it as an untrusted dependency rather than a finished asset.
• Baseline normal AI behavior before enforcing detection Define expected CPU, GPU, memory, and network patterns for approved training and inference jobs, then alert on departures from those profiles. This makes it possible to catch poisoned execution, unusual inter-node communication, and suspicious process chains before the model finishes.
• Revoke and revalidate automation credentials on a short leash Inventory legacy tokens, service accounts, and agent permissions that can still publish packages, update pipelines, or trigger model deployments. Remove access that no longer maps to an active business purpose, and require explicit revalidation for privileged automation paths.

Key takeaways

• AI workloads expose a mismatch between deterministic HPC security models and the dynamic behaviour of modern model pipelines.
• Trusted delivery channels do not guarantee trusted outcomes when the real risk sits in model artifacts, automation credentials, and runtime behaviour.
• Practitioners should move from static approval checks to continuous identity, provenance, and behavioural assurance across the AI execution lifecycle.

Key terms

• Model Provenance: Model provenance is the traceable history of a model’s origin, training data, checkpoints, and deployment path. In AI security, it is the evidence that tells you whether a model artifact can be trusted, verified, and governed before and during use.
• Runtime Trust: Runtime trust is the assumption that a workload remains safe after it starts executing. For AI and HPC environments, that assumption is weak unless behaviour, access, and artifacts are continuously validated while the workload is active.
• Behavioral Baseline: A behavioral baseline is the expected pattern of activity for a workload, including compute, memory, process, and network behaviour. It allows defenders to spot when AI training or inference deviates from approved operation, even if the code or package appears legitimate.
• Supply Chain Artifact: A supply chain artifact is any trusted component that enters an environment through a legitimate delivery path, such as a package, token, model checkpoint, or signed binary. In AI systems, the term extends beyond software to the data and model layers that shape outcomes.

What's in the full article

SentinelOne's full article covers the operational detail this post intentionally leaves for the source:

• Technical examples of the AI supply chain attacks against LiteLLM, Axios, and CPU-Z.
• The specific runtime monitoring architecture proposed for AI workloads on HPC systems.
• Discussion of behavioral AI detections and kernel-level monitoring considerations.
• The article's citations and research references on poisoning, side channels, and inference compromise.

👉 The full SentinelOne article covers the technical proposal, attack examples, and cited research behind the governance gap.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.