AI workloads expose the gaps in HPC security overlays

At a glance

What this is: This is SentinelOne's analysis of why NIST SP 800-234's HPC security overlay does not fully cover AI workloads, especially where deterministic assumptions fail under dynamic training, inference, and supply-chain risk.

Why it matters: It matters because practitioners running AI on HPC infrastructure need to reassess whether controls built for predictable compute can govern model training, shared accelerators, and runtime integrity.

By the numbers:

• The US Federal Government is committing $600 million to build one of the world’s most advanced AI infrastructure systems.
• NIST SP 800-234 tailors 60 controls across four security zones for HPC workloads.

Context

AI workloads are computed differently from classic HPC jobs, and that difference matters for security architecture. NIST SP 800-234 was built for deterministic workloads that behave predictably, while AI training and inference can change behaviour, execution paths, and artefact integrity during runtime.

The governance gap is not just a scaling issue. When a control model assumes static inputs, predictable execution, and clean hand-offs between jobs, it can miss poisoning, side-channel exposure, and model integrity drift across the training-to-serving lifecycle.

Key questions

Q: What breaks when AI workloads are governed like deterministic HPC jobs?

A: The main failure is assumption mismatch. Deterministic HPC controls assume predictable execution, clean job boundaries, and integrity checks that happen before or at load time. AI workloads can change state during training and inference, so those controls often miss poisoning, side channels, and runtime drift that only appear after the job has started.

Q: Why do AI supply chain attacks bypass traditional HPC security controls?

A: Because the channel can be trusted even when the intent is not. Signed packages, legitimate repositories, and valid access tokens can all deliver malicious artefacts that pass ordinary authorization checks. For AI environments, that means provenance and runtime verification matter as much as perimeter trust.

Q: How do teams know whether runtime monitoring for AI on HPC is effective?

A: Look for telemetry that can detect behaviour changes during training or inference, not just at job start. Effective monitoring should surface unexpected process chains, abnormal GPU and memory patterns, unusual inter-node communication, and model-output shifts that indicate a poisoned or modified workflow.

Q: Which controls matter most when AI runs on shared accelerator infrastructure?

A: The highest-value controls are workload isolation, continuous behavioural telemetry, artefact provenance, and integrity verification during execution. Shared accelerators create attack surfaces that do not exist in single-tenant compute, so controls must address co-tenancy, side channels, and runtime trust boundaries.

Technical breakdown

Why deterministic HPC controls fail under AI workloads

NIST SP 800-234 tailors controls for high-performance compute jobs that are expected to run repeatably under known inputs. That model works when the security problem is perimeter access, job isolation, memory clearing, and integrity at load time. AI workloads are different because the security-relevant state emerges during training and inference, not just at job start. Poisoned data can look statistically normal, and a compromised model can behave correctly until a trigger appears. The overlay was not written to observe that kind of runtime variability.

Practical implication: Treat AI training and serving as runtime security problems, not only load-time integrity problems.

Why supply chain assurance is incomplete for AI artefacts

The article distinguishes conventional software supply chain controls from the AI-specific chain of training data, checkpoints, and model weights. Traditional SR controls can help with software and hardware provenance, but they do not fully describe the trust boundary for pre-trained models, poisoned datasets, or malicious artefacts that arrive through legitimate repositories. In AI environments, the attacker may subvert intent while preserving authorization, which makes signature checks and trusted channels necessary but insufficient.

Practical implication: Extend supply chain governance to training data provenance, model weight integrity, and validated model ingestion.

How shared accelerators and runtime monitoring become attack surfaces

AI on shared GPU infrastructure creates exposure during active execution, not only at transition points. Co-tenant workloads, side channels, and anomalous compute behaviour can leak information or signal compromise while training is still underway. The article also argues that co-scheduled monitoring can be starved or manipulated, so monitoring has to be isolated from the workload it watches. This is why behavioural telemetry at the training layer matters more than perimeter-only visibility.

Practical implication: Separate security monitoring from the workload plane and baseline normal GPU, memory, and network behaviour continuously.

Threat narrative

Attacker objective: The objective is to influence model behaviour, steal infrastructure data or secrets, and persist inside AI delivery pipelines while appearing legitimate.

1. Entry via trusted software supply chain channels allowed malicious packages or binaries to reach AI development and HPC environments without initial suspicion.
2. Credential or artefact abuse followed when stolen tokens, legacy access, or legitimate signing paths let malicious payloads pass authorisation checks.
3. Impact emerged as poisoned training, covert GPU side channels, or modified inference behaviour that conventional perimeter and signature controls could not reliably detect.

Breaches seen in the wild

• ShadowRay 2024: Attackers took over internet-exposed Ray AI clusters via the disputed CVE-2023-48022, exposing cloud keys, AI API tokens and GPU compute.
• AI LLM hijack breach: attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI workloads expose an identity assumption collapse in HPC security: the overlay assumes workloads behave predictably long enough for perimeter controls, load-time attestation, and memory clearing to be meaningful. That assumption fails when the actor is an AI workload that changes its security-relevant state during training or inference. The implication is that governance has to be built around runtime behaviour, not just controlled entry points.

Model-layer provenance is now part of the trust boundary: software supply chain controls alone do not describe the security problem when the artefact being ingested is a pre-trained model, a checkpoint, or poisoned training data. The attack path can remain fully authorised while the intent is malicious. Practitioners should treat model provenance as a first-class governance object, not as an extension of ordinary package management.

Shared compute changes the blast radius of compromise: on multi-tenant accelerator infrastructure, the risk is not only theft of compute cycles but information leakage through side channels and cross-workload interference. That makes isolation, telemetry, and workload placement policy more than infrastructure preferences. They become core identity and trust controls for AI execution.

AI supply chain trust debt: the article points to a growing mismatch between trusted delivery and untrusted intent. A model or package can arrive through a legitimate path and still carry behaviour that subverts the intended workflow. For practitioners, the issue is not whether a channel is signed, but whether the thing delivered is governed for AI-specific misuse.

HPC security overlays need AI-specific control extensions: the current framework can carry conventional HPC protections, but it does not fully express what has to be verified inside training loops, inference pipelines, and model artefact lifecycles. That gap does not make the overlay obsolete, but it does make it incomplete for AI-native deployments. Security teams should treat this as a governance boundary shift, not a tuning problem.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Infrastructure Workload Identity Guide

What this signals

AI workloads expose a control boundary, not just a capacity problem: once model training and inference become security-relevant runtime events, controls built for predictable HPC jobs stop providing full assurance. That shifts programme design toward telemetry, provenance, and execution-time integrity rather than relying on perimeter scanning and job transition checks.

AI infrastructure trust debt: the combination of legitimate delivery paths and malicious intent creates a gap that ordinary signature and reputation systems cannot close. In practice, that means your identity and workload governance programme has to assume that some trusted channels will still deliver untrusted artefacts.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That reinforces the operational reality that AI infrastructure is now a primary exposure surface, not a peripheral one.

For practitioners

• Map AI workloads to a separate runtime control set Classify training and inference workloads separately from deterministic HPC jobs and identify where perimeter scanning, memory clearing, and load-time attestation stop giving you meaningful assurance.
• Add provenance checks for model artefacts Require traceability for training data, checkpoints, and pre-trained models so ingestion is governed as a model-lifecycle activity, not just a software package event.
• Isolate security monitoring from the workload plane Reserve dedicated compute or other out-of-band monitoring capacity so telemetry cannot be starved, filtered, or co-opted by the AI job under observation.
• Baseline behaviour for training and inference Establish expected GPU utilisation, memory access patterns, and inter-node communication profiles so deviations can be detected while the model is still executing.
• Review trusted-channel assumptions in your software supply chain Audit repositories, signing paths, and token revocation processes to confirm that legitimate delivery channels cannot be used to publish or run malicious artefacts.

Key takeaways

• AI workloads change the security problem from predictable execution to runtime behaviour that can drift, poison, or conceal compromise inside otherwise legitimate jobs.
• The article shows that trusted channels, signatures, and perimeter checks are not enough when artefacts, models, or tokens can be abused without breaking the authorization chain.
• Practitioners should move governance toward provenance, isolated monitoring, and execution-time integrity controls if they want meaningful assurance for AI on HPC.

Key terms

• AI workload: A computing job that trains, serves, or otherwise runs machine learning models, often with dynamic behaviour that changes during execution. In security terms, AI workloads need controls for runtime integrity, provenance, and telemetry because the model, data, and execution path can all become part of the attack surface.
• HPC security overlay: A control layer that adapts baseline security requirements to high-performance computing environments. It is designed to preserve performance while addressing shared resources, job isolation, and integrity concerns, but it may assume more predictable execution than AI workloads actually provide.
• Model Provenance: Model provenance is the evidence chain showing where an AI artefact came from, how it was modified, and whether the version in use is the one that was approved. For AI security teams, provenance is the control that turns trust from assumption into verification.
• Runtime Integrity: Runtime integrity is the assurance that what a system executes matches what defenders expect at the moment of execution. It matters because malware or exploit code can change memory, cache, or process behaviour without changing the on-disk file, which makes artifact-based checks incomplete on their own.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.