Operationalizing AI security for workloads where models run

At a glance

What this is: Aqua Security’s article argues that AI security has to move from model-level discussion to runtime protection for the workloads where AI actually runs, with visibility, policy enforcement, and response built into production environments.

Why it matters: For IAM, security architecture, and governance teams, the implication is that AI controls need to sit at the workload layer, not just around prompts or application design, if they are to be enforceable in production.

By the numbers:

• Over 70% of AI applications are deployed in containers, running on Kubernetes and cloud native infrastructure, according to Aqua Security.
• 97 percent of security organizations plan to increase spending on securing AI use cases, according to Aqua Security.

Context

AI workload security is the control problem that emerges when models move from experimentation into production services. At that point, the question is no longer whether AI exists in the environment, but where it runs, what it touches, and whether policy can be enforced at runtime.

For identity and security teams, this shifts the centre of gravity from user-facing AI controls to the infrastructure layer that hosts inference, data handling, and application execution. Without workload-level visibility, organisations can neither govern AI usage consistently nor prove that policy is being applied where decisions are made.

Key questions

Q: How should security teams govern in-house AI inference workloads?

A: Security teams should govern in-house AI inference workloads as non-human identities with scoped permissions, named ownership, and lifecycle controls. That means inventorying service accounts, separating deployment rights from infrastructure administration, and reviewing credentials whenever models move between environments or vendors. If the workload can call other systems, it needs a defined trust boundary.

Q: Why do AI firewalls and SDK controls leave gaps in production governance?

A: They sit too far from the execution layer to guarantee complete coverage. Those controls may inspect prompts or enforce application rules, but they do not automatically provide central visibility across the workload lifecycle where models actually run and policy drift can occur.

Q: What are the signs that AI security controls are not reaching the runtime layer?

A: A common sign is that teams can describe AI usage at the application level but cannot say which model is running, where it is deployed, or how policy is enforced during execution. That gap means the organisation is observing AI, not governing it.

Q: Should organisations prioritise workload-level AI security over prompt-level controls?

A: Yes, when the production risk comes from how models run inside containerised services. Prompt-level controls can help at the edge, but workload-level security is what determines whether policy can be enforced where inference, data access, and execution decisions actually happen.

Technical breakdown

Why AI workload security depends on runtime visibility

AI workloads in production are not governed effectively by static configuration review alone. The article’s core point is that security teams need to see which models are running, how prompts are handled, and whether usage aligns with policy while execution is happening. That requires telemetry at the workload layer, not just application logs or user-facing gateways. Aqua’s approach uses an eBPF-based agent inside the workload, which is a mechanism for observing kernel-level activity without modifying application code. The architectural significance is that the control point moves closer to where inference, sensitive data access, and policy decisions occur.

Practical implication: place detection and policy enforcement where AI workloads execute, not only where users interact with them.

Why AI firewalls and SDK controls leave a governance gap

The article distinguishes between controls that inspect user interactions and controls that govern the infrastructure where AI logic runs. AI firewalls can help with prompt-side monitoring, and SDKs can enforce application-level rules, but neither necessarily provides centralised visibility across the full workload lifecycle. That matters because production risk sits in execution context, not in a single interface. If a team can see only one layer, it may miss model usage, prompt handling, or policy drift elsewhere in the stack. The technical issue is coverage: controls that are too far from execution do not reliably govern the runtime environment.

Practical implication: test whether each control actually covers the workload lifecycle, not just the application edge.

Why containerised AI changes the identity and policy boundary

Aqua Security cites Gartner’s observation that most AI applications are deployed in containers on Kubernetes and cloud native infrastructure. That matters because these environments already rely on workload identity, runtime segmentation, and policy enforcement patterns that can be extended to AI. AI is therefore not a separate security island. It becomes another production workload with data access, execution constraints, and observable runtime behaviour. The architecture question is whether existing platform controls can be adapted to the AI layer without breaking developer velocity or creating blind spots in governance.

Practical implication: map AI workloads into the same cloud-native governance model you use for other production services.

NHI Mgmt Group analysis

AI workload governance fails when security stops at the interface. The article shows that prompt-side controls and application SDKs do not fully govern the environment where inference and data handling occur. That creates a runtime governance gap rather than a simple detection problem. Practitioners should treat the workload layer as the real enforcement boundary.

Runtime visibility is the decisive control variable for production AI. Once models run inside containerised services, the security question becomes whether policy can be observed and enforced during execution. That is the point where access to data, prompt handling, and model behaviour become operational concerns rather than design assumptions. Teams should measure control coverage at runtime, not just at development time.

AI in containers should be treated as cloud-native infrastructure, not a special case. The article’s container and Kubernetes emphasis reinforces that AI security inherits the same operational reality as other production workloads. That means identity, segmentation, logging, and policy enforcement remain the governing disciplines, but they must be applied at the workload layer. Security leaders should align AI controls with existing cloud-native governance rather than building a parallel model.

Operationalising AI security is now a policy enforcement problem, not a model evaluation problem. The article’s programme framing shows a market shift from proving AI exists to proving AI behaves within approved limits in production. That is why governance, observability, and runtime control are converging around the workload. Practitioners should reframe AI security programmes around enforcement points that can survive scale.

Prompt monitoring alone does not create governance certainty. The article makes clear that security value comes from seeing what models are used, how prompts are processed, and whether policy is being followed at the point of execution. That broader control plane is what turns AI from an unmanaged capability into a governed production asset. Security teams should evaluate controls by what they can constrain in runtime, not by what they can inspect in transit.

From our research library:

• Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.
• Read next: AI Infrastructure Workload Identity Guide

What this signals

AI workload security is becoming a runtime governance discipline. As AI moves into production services, the practical question is whether security teams can enforce policy where models execute, not only where users interact with them. That shifts AI security into the same operational conversation as cloud-native workload governance and identity-aware access control.

Workload identity is the missing bridge between AI governance and cloud-native operations. If AI runs inside containers and Kubernetes, the security model has to reflect the same service-to-service control logic already used elsewhere in the stack. Teams that align AI controls to workload identity and runtime policy will have a more realistic path to enforcement than those relying on edge inspection alone.

For practitioners

• Define the AI workload as the control boundary Document which production services host inference, handle prompts, or touch sensitive data, then assign those services to your security governance model. This prevents AI risk from being treated as a generic application issue.
• Validate runtime policy enforcement coverage Test whether controls can block or allow AI behaviour while execution is happening, not just report on it after the fact. Focus on prompt handling, model usage, and data access inside the workload.
• Map AI workloads to existing cloud-native protections Reuse container, Kubernetes, and workload identity patterns where they already exist, but verify they cover the AI execution path rather than only the surrounding platform. This reduces blind spots between application and infrastructure teams.
• Separate edge controls from runtime governance Treat AI firewalls and SDK-based controls as partial safeguards, not complete governance. Use them for inspection and input control, but keep a separate control objective for the workload where model execution occurs.

Key takeaways

• AI security in production is no longer mainly about model experimentation. The harder problem is governing the workloads where inference, prompts, and data access happen.
• The article points to a runtime gap, not just a visibility gap. Security teams need controls that can observe and enforce policy inside production services.
• AI workloads should be governed like cloud-native production services. That means aligning visibility, identity, and policy enforcement to the execution layer.

Key terms

• AI Workload Security: AI workload security is the practice of protecting systems that run autonomous or semi autonomous AI agents, models, and supporting services. It focuses on runtime behaviour, tool use, data access, and policy enforcement, because these workloads can act unpredictably in ways that traditional container security cannot fully observe or control.
• Runtime Policy Enforcement: Runtime policy enforcement evaluates a request at the moment it is executed instead of relying only on preconfigured permissions. For AI agents, this allows decisions to reflect current context, target sensitivity, and behavioural signals rather than static assumptions.
• Workload Visibility Gap: A mismatch between the speed of ephemeral infrastructure and the organisation’s ability to observe it. When workloads are short-lived, logs, inventories, and manual reviews can miss the exact traffic or privilege path that an attacker uses.
• Cloud-Native AI Security: Cloud-native AI security is the application of container, Kubernetes, and runtime controls to AI systems running as production services. It treats AI as part of the operational stack, so identity, segmentation, logging, and enforcement can be applied consistently.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.