Modal shows why AI infrastructure needs compute-native identity controls

At a glance

What this is: This interview argues that AI infrastructure built for bursty GPU workloads changes the identity boundary, because static cloud and Kubernetes assumptions break when workloads scale and move dynamically.

Why it matters: IAM, PAM, and NHI teams need to govern the workload itself, not just the platform it runs on, because elasticity changes where access, isolation, and accountability have to be enforced.

Context

AI infrastructure is the compute and orchestration layer that runs model training and inference across GPUs, containers, and cloud capacity. In this interview, WorkOS frames the problem as one of control mismatch: traditional infrastructure abstractions were not designed for the speed, scale, and burstiness of modern AI workloads.

That matters for identity governance because GPU-backed workloads, container startup, and cloud scheduling decisions change when and how workload identity is asserted. If access controls assume stable servers, fixed runtime patterns, or slow-moving infrastructure, they will not describe the real control boundary for AI systems.

Key questions

Q: How should teams govern AI workloads that scale up and down in seconds?

A: Treat the workload as the identity subject, not the server or cluster. Define what the job may access at launch, bind permissions to the runtime phase, and expire entitlements when the compute session ends. If governance still depends on slow review cycles, it will miss the actual risk window.

Q: Why do bursty GPU workloads break traditional IAM assumptions?

A: Because the control model often assumes long-lived infrastructure and stable access paths. Bursty AI jobs can request compute, load models, and finish before a human review cycle would ever see them, so entitlement management has to move closer to issuance and runtime enforcement.

Q: What are the signs that AI workload isolation is too tied to infrastructure?

A: Look for permissions that follow the environment instead of the job, persistent entitlements after container teardown, and approval processes that cannot keep pace with short-lived inference sessions. Those are symptoms that the control boundary is still anchored to static infrastructure rather than dynamic workload behaviour.

Q: How do workload identity and runtime isolation complement each other for AI systems?

A: Workload identity decides what the AI job is allowed to do, while runtime isolation constrains where and how it can do it. Used together, they reduce the chance that a fast-starting GPU job can exceed its intended scope even when infrastructure is highly elastic.

Technical breakdown

Why bursty GPU workloads strain static infrastructure assumptions

AI inference often arrives in spikes, not as steady traffic. That means capacity, scheduling, and container lifecycle decisions must happen dynamically, sometimes within seconds, across distributed GPU pools. Traditional infrastructure models assume more predictable placement and longer-lived runtime states, which is why they become awkward when AI systems need to scale up, spin down, and relocate quickly. In identity terms, the workload is no longer anchored to a single machine or a long-lived process, so the trust boundary must move with the workload and its execution context.

Practical implication: govern access at workload issuance and runtime, not only through static environment-level entitlements.

How compute-native identity changes workload isolation

The article notes that Modal uses gVisor for workload isolation while replacing much of the operational burden of Kubernetes and Docker. That points to a broader pattern in AI infrastructure: isolation cannot rely only on network segmentation or cluster architecture if the runtime itself is elastic and ephemeral. Identity for AI workloads therefore has to express who or what is allowed to start compute, load models, and consume GPU capacity under specific conditions. The control problem becomes one of binding authorization to execution context, not just to an account or a cluster role.

Practical implication: align workload identity, runtime isolation, and authorization decisions so GPU jobs cannot exceed their intended execution context.

Why container speed changes the control boundary

Fast container startup matters because it shortens the time between request, execution, and teardown. In an AI infrastructure environment, that can make traditional approval, review, or recertification cadences too slow to capture meaningful risk. The article's key infrastructure point is that the platform exists to make GPU-heavy workloads feel fluid, which also means access and observability must be engineered for short-lived compute. Security teams should treat rapid startup as an identity design constraint, because the lifecycle of the workload is now part of the trust model.

Practical implication: design controls that follow container lifecycle events and GPU job duration, not fixed administrative review cycles.

NHI Mgmt Group analysis

Compute-native identity is now a control requirement, not an optimisation. When AI infrastructure can load models, start containers, and claim GPU capacity in seconds, the identity model must track the workload at runtime rather than the server it lands on. That changes the governance unit from environment to execution, which is why AI infrastructure teams need workload identity discipline, not just infrastructure orchestration. The practitioner conclusion is simple: static platform assumptions no longer describe the real access boundary.

Static infrastructure assumptions fail when execution is elastic. Traditional cloud and Kubernetes controls were built around more durable runtime states, where a machine, namespace, or node could be reviewed after allocation. Bursty inference and rapid scaling break that assumption because the asset of interest is a short-lived compute session, not a persistent host. The implication is that access governance has to move closer to issuance and runtime enforcement.

Workload identity becomes the only stable anchor when GPUs are fungible. The article highlights that AI applications need capacity across the world and can consume it in very short bursts. That means the control question is not which GPU owns the session, but which workload is entitled to consume which compute under which conditions. Identity blast radius: when compute is elastic, the scope of compromise or misuse follows the workload boundary, not the physical infrastructure boundary. Practitioners should treat workload identity as the durable policy surface.

AI infrastructure governance is converging with NHI governance. These systems are non-human identities in motion: they authenticate, request capacity, and execute without human pacing. That puts them in the same governance class as service accounts and other machine identities, but with more dynamic runtime behaviour. The field should expect greater pressure to align NHI inventory, workload identity, and runtime enforcement, because infrastructure abstractions alone cannot express who is really using the GPU estate.

The category is moving from platform abstraction to policy precision. Modal's thesis shows that the market is rewarding infrastructure that makes AI execution easier, but ease of execution also raises the governance bar. As AI adoption spreads from startups to digital natives and eventually large enterprises, security teams will need finer-grained policy around who can launch compute, where models run, and how quickly runtime access expires. The practitioner conclusion is to make policy follow the workload lifecycle, not the infrastructure brand.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Infrastructure Workload Identity Guide

What this signals

Compute-native identity: AI infrastructure teams are being forced to treat workload execution as an identity problem, because scale events and GPU bursts now happen faster than conventional governance cycles. That means access reviews, namespace design, and entitlement scope all need to be evaluated against short-lived compute rather than persistent servers.

When AI workloads can consume GPU capacity globally and change state in seconds, the practical boundary for control becomes the model-serving job itself. Teams that keep anchoring policy to clusters and cloud accounts will miss the point where access actually occurs, so the programme should shift toward runtime-aware workload identity.

This pattern also puts pressure on NHI governance programmes to absorb AI services into the same lifecycle discipline used for machine identities. The operational question is no longer whether a workload is automated, but whether it can be governed at the moment it starts, scales, and exits.

For practitioners

• Map AI workloads to their true identity boundary Inventory which AI jobs, model runners, and inference services act as non-human identities, then document what they are allowed to start, read, and scale. Treat the workload lifecycle as the object of governance rather than the cluster or cloud account alone.
• Bind authorization to execution context Require that GPU-intensive jobs receive only the permissions needed for the specific model, dataset, and runtime phase they are executing. Where possible, make entitlements expire with the job so access cannot outlive the compute session.
• Review runtime isolation for elastic workloads Check whether your container isolation, namespace boundaries, and node assumptions still hold when workloads start and stop rapidly across regions and GPU pools. If they do not, redesign the control point around the runtime rather than the infrastructure layer.
• Align NHI governance with AI deployment cadence Add AI services to NHI lifecycle, recertification, and offboarding processes so the control model covers model-serving jobs, batch inference, and ephemeral compute agents. Use shorter review intervals for high-velocity workloads that can scale in minutes.
• Separate provisioning from persistence Ensure that the ability to launch GPU capacity does not imply standing access to adjacent data stores, tooling, or orchestration layers. The fewer persistent grants a workload retains after startup, the smaller the blast radius if the runtime is misused.

Key takeaways

• AI infrastructure for bursty GPU workloads changes the identity boundary because the workload, not the server, becomes the control subject.
• Traditional cloud and Kubernetes assumptions break down when AI jobs start and stop faster than review cycles can observe them.
• Practitioners should govern access at runtime, align isolation with workload lifecycle, and bring AI services into NHI lifecycle controls.

Key terms

• Compute-native identity: A governance model that treats the AI workload or compute job as the identity subject, not only the host, cluster, or cloud account. It matters when runtime behaviour changes quickly and access must be controlled at the point of execution, especially for elastic GPU-driven systems.
• Workload Identity: The identity assigned to a software workload, such as a containerised application, serverless function, or microservice, enabling it to authenticate to other services without storing static credentials.
• Runtime Isolation: A containment pattern that separates risky execution from the main application so compromise has limited blast radius. For AI systems, it is the practical control that keeps a manipulated model response from inheriting full application or infrastructure privileges.
• Identity Blast Radius: The amount of damage a compromised identity can cause across systems, data, and infrastructure. In NHI environments, it is shaped by permissions, network reach, and administrative capability rather than by the credential alone. Reducing blast radius is a containment strategy that limits lateral movement and data exposure.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.