Open source Kubernetes AI stacks need packaging and policy

At a glance

What this is: Jozu maps the open source Kubernetes AI stack into six layers and argues that packaging and policy are the most common governance gaps.

Why it matters: IAM, NHI and platform teams need a versioned, policy-enforced model supply chain because Kubernetes AI stacks fail when artifacts, permissions and runtime controls are managed separately.

Context

Enterprise AI on Kubernetes is usually assembled from multiple open source projects rather than delivered as one coherent platform. That creates a governance problem as well as an architecture problem, because teams must coordinate model development, scheduling, serving, packaging and enforcement across different control points.

The first practical question is not which project is best, but where the stack has broken ownership. When the model, data, code and runtime policy are managed in different systems, the organisation loses traceability over what is deployed and who can approve it, which is exactly where AI infrastructure workload identity and policy controls start to matter.

Key questions

Q: What breaks when AI model evaluations are not versioned?

A: Without versioned datasets and prompts, score changes become hard to trust because the difference may come from the test setup rather than the model. That makes comparisons unreproducible and obscures whether a regression was caused by prompt drift, input changes, or model behavior. Version control preserves traceability and makes later re-testing meaningful.

Q: Why does AI governance depend on signed provenance as much as cluster policy?

A: Cluster policy can only enforce rules on what it can recognise, so it cannot replace trust in the underlying artefact. Signed provenance gives the platform a verifiable basis for deciding whether a model, package or dependency is approved. Without that, admission control becomes a shallow filter, and teams end up relying on manual review after the workload is already assembled.

Q: How should security teams govern AI in the security stack?

A: Security teams should treat AI as a governed decision aid, not an autonomous authority. Define where it can assist detection, prioritisation, and enrichment, then require human or policy approval for privileged actions and access decisions. The key control is traceability, so every AI-supported recommendation can be reviewed, challenged, and overridden.

Q: Should organisations prioritise model packaging before adding more serving tools?

A: Yes, when the organisation cannot reliably answer which model version is in production or how its dependencies were approved. Packaging fixes the traceability problem first, which makes later serving, scheduling and policy decisions meaningful. Adding more serving layers before provenance is in place usually increases speed without improving governance.

Technical breakdown

Why Kubernetes AI stacks fragment into six layers

A Kubernetes AI stack usually separates into development, scheduling, distributed compute, serving, packaging and policy because each layer solves a different operational problem. Kubeflow handles notebooks and workflows, Kueue or Volcano manage scarce GPU capacity, Ray scales distributed jobs, KServe or vLLM handle inference, and policy engines enforce admission rules. The fragmentation is practical, but it also means no single component has a complete view of the model lifecycle. That leaves governance to stitching, not design. Practical implication: treat the stack as one governed system, not a set of interchangeable tools.

Practical implication: Define ownership across the full AI lifecycle, not just per tool, so security decisions do not stop at the cluster boundary.

Why model packaging becomes a security control

Packaging is the layer that turns an ad hoc set of model files, datasets, code and prompts into a traceable artifact. Without that, teams cannot prove which inputs produced a deployed model, cannot roll back cleanly, and cannot apply consistent signing or verification. KitOps uses OCI artifacts so the same registry and supply chain mechanics used for containers can carry the model unit as well. That matters because AI governance fails when lineage is implicit instead of enforced. Practical implication: make versioned model artifacts the unit of approval, not the notebook or the file share.

Practical implication: Approve and deploy only versioned model artifacts so provenance, rollback and scanning are attached to the same object.

How admission policy and identity shape AI runtime trust

Policy engines such as Kyverno and OPA/Gatekeeper control what the cluster will admit, which makes them the final enforcement point before runtime. That is useful, but only if identity and signing are already in place, because policy can only evaluate what it can verify. The deeper issue is that general Kubernetes policy does not understand model semantics, agent behaviour or AI-specific payload risk. Practical implication: combine cluster admission with identity-bound approval and runtime scanning so the policy layer is not asked to solve provenance alone.

Practical implication: Tie admissions to signed artifacts and governed identities so the cluster can reject unapproved AI workloads before execution.

Threat narrative

Attacker objective: The objective is to get manipulated or untrusted AI artefacts into production workflows while preserving enough legitimacy to evade governance and detection.

1. Entry occurs when AI artefacts, model code or dependency packages are introduced through loosely governed repositories or registries.
2. Credential or artifact trust is then abused when unsigned or untracked model units reach training or serving workflows without enforceable provenance checks.
3. Impact follows when teams cannot prove what is running, roll back safely, or stop a compromised model from reaching production inference.

Breaches seen in the wild

• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Open source AI governance fails first at the seams, not in the model itself. The article shows that enterprises rarely run a single platform, so control is distributed across orchestration, scheduling, serving, packaging and policy. That creates a governance seam where ownership can be lost between teams, especially when identity, signing and approval are handled outside the AI stack. The practical conclusion is that the control objective is lifecycle coherence, not tool consolidation.

Model packaging is the most underrated identity problem in AI infrastructure. Once a model, its datasets, code and configuration are detached from one another, the organisation loses the ability to bind approval to a specific artefact state. That is a provenance problem, but it also becomes an identity problem because the cluster is asked to trust artefacts whose origin and authority are no longer explicit. In practice, the security boundary should move from the file to the signed, versioned unit of deployment.

Policy engines are necessary but not sufficient for AI runtime governance. Kyverno and OPA can decide what enters the cluster, but they cannot infer whether a model file is malicious, whether an agent should be allowed to act, or whether a dependency chain is semantically safe. That means policy has to sit on top of trustworthy identity and artefact controls, not replace them. Practitioners should treat admission as the last gate, not the primary assurance mechanism.

AI infrastructure workload identity is becoming the hidden control plane for model operations. The article points to the need to integrate identity and logging, which is where AI infrastructure workload identity becomes central to the operating model. If the platform cannot clearly distinguish which workload, pipeline or service is authorised to move a model forward, then governance becomes manual review after the fact. The result is that identity and provenance must be designed together, not added later.

Open source improves composability, but it also shifts assurance work onto the enterprise. The ecosystem provides strong building blocks, yet the article is right that the open source stack does not by itself answer scanning, runtime governance or malicious model detection. That means security leaders need an explicit assurance model for AI infrastructure rather than assuming Kubernetes primitives are enough. The practical conclusion is to define who verifies artefacts, who approves runtime use and who can revoke trust.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Enterprise AI use rose from 55% of organisations in 2023 to 88% in 2025, according to McKinsey’s Global Surveys on the State of AI.
• Read next: AI Infrastructure Workload Identity Guide

What this signals

Model provenance is becoming the control point that determines whether AI operations are governable at all. As Kubernetes AI stacks grow more modular, teams need to stop treating packaging as a convenience layer and start treating it as the record of authority for model deployment. Without that shift, policy, logging and scheduling remain disconnected controls instead of one assurance chain.

The next governance step is to align cluster policy, signing and workload identity around the same artefact lifecycle, because the control that verifies a model must also know which workload is allowed to move it. That is where AI infrastructure governance starts to resemble supply chain governance, but with runtime identity attached to every promotion decision.

Provenance without runtime identity is incomplete: a signed model is still not safe if the workload that fetches, unpacks or serves it is not itself governed. Enterprises should therefore design for both artefact trust and execution trust, and use AI infrastructure workload identity as the bridge between them.

For practitioners

• Map the AI stack as one governed lifecycle Document where notebooks, training jobs, schedulers, serving endpoints, registries and policy decisions live, then assign an owner for each transition between them.
• Package models as versioned artefacts before promotion Require the model, datasets, code, prompts and configuration to move together as a single approved unit so lineage and rollback stay intact.
• Bind deployment approval to signed provenance Use signature verification on OCI-stored model artefacts and refuse promotion when provenance, attestation or ownership cannot be verified.
• Separate scheduling decisions from serving trust Treat GPU queueing and model serving as different governance problems, because capacity fairness does not prove artifact safety or runtime legitimacy.

Key takeaways

• The article’s central point is that enterprise AI on Kubernetes is usually a stitched-together stack, so governance has to span development, scheduling, serving, packaging and policy.
• The most consequential gap is provenance: without a versioned model artefact, teams cannot reliably prove what is running, how it was approved or how to roll it back.
• Practitioners should put packaging and signed approval ahead of platform sprawl, because those controls make the rest of the AI stack governable.

Related NHI Mgmt Group resources: AI Infrastructure Workload Identity Guide · Open Source Security Guide · Agentic AI Security Policy Template · AI Governance Platform Buyer's Guide

Key terms

• ModelKit: A ModelKit is a packaged AI artefact that bundles a model with the datasets, code, prompts and configuration needed to reproduce or deploy it. The goal is to make the model unit versioned, portable and auditable so provenance and rollback are attached to one controlled object.
• Model Provenance: Model provenance is the evidence chain showing where an AI artefact came from, how it was modified, and whether the version in use is the one that was approved. For AI security teams, provenance is the control that turns trust from assumption into verification.
• Admission Controller: An admission controller is a Kubernetes control that validates or changes workload requests before the cluster admits them. It acts as a deployment-time policy layer, which makes it useful for blocking unsafe images, rejecting risky configuration, and enforcing runtime standards that build-time scans may miss.
• Workload Identity: The identity assigned to a software workload, such as a containerised application, serverless function, or microservice, enabling it to authenticate to other services without storing static credentials.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security and workload identity. It helps practitioners connect identity controls to the broader security architecture their programmes depend on.