Confidential AI closes the in-use gap for regulated enterprise AI

At a glance

What this is: This is an analysis of confidential AI as a hardware-backed approach to protecting data and model weights during inference, with the central finding that the in-use gap is what blocks many enterprise deployments.

Why it matters: It matters because IAM, NHI, and AI security teams need a control model that protects sensitive data and model access while workloads are actively running, not only at rest or in transit.

Context

Confidential AI addresses a governance gap that traditional data security controls do not close: once sensitive inputs are decrypted for processing, the operating environment itself becomes part of the trust boundary. In AI deployments, that means the risk is not limited to storage or transport, but extends into memory, privileged infrastructure access, and model exposure.

That matters for regulated sectors because the security question changes from whether data is encrypted to whether the computation environment can be trusted while it uses the data. The same problem applies to proprietary models, which can be extracted if they are loaded into an untrusted runtime. In practice, this is a cloud and AI governance issue with a real identity dimension, because privileged administrators and workload identities sit inside the control path.

Key questions

Q: How should organisations protect the data that powers AI workloads across training, inference, and retrieval systems?

A: Security teams should treat AI data as a production dependency, not a side asset. Protect training sets, inference inputs, and retrieval databases with strong access controls, encryption, backup, immutability, and recovery testing. The goal is to preserve confidentiality, integrity, and availability so corrupted, deleted, or encrypted data does not break models, distort outputs, or interrupt business operations.

Q: Why do confidential AI controls matter more than storage encryption for enterprise AI?

A: Storage encryption protects data before and after use, but inference requires the data to exist in memory while the model runs. That is when host-level access, privileged administrators, and infrastructure operators can become the real exposure path. Confidential AI addresses that moment of use, which is where many enterprise risk decisions are actually made.

Q: What breaks when AI workloads run on infrastructure that cannot be attested?

A: The trust model breaks because the organisation has no cryptographic proof that the runtime is authentic and unmodified. Without attestation, key release becomes a matter of policy trust rather than verified control. That leaves sensitive inputs and model assets exposed to the very environment that is processing them.

Q: Should teams prioritise confidential AI before expanding agentic AI in regulated environments?

A: Yes, when agentic systems will handle sensitive records, proprietary models, or continuous internal access. Agentic workflows increase the number of actions, tools, and persistence points that must be trusted. If the runtime cannot be verified and isolated, scaling agentic AI expands exposure faster than governance can absorb it.

Technical breakdown

Why the in-use gap matters for AI workloads

Most security models are strongest when data is at rest or in transit, because encryption cleanly separates the attacker from the asset. During inference, that separation disappears: the workload must see plaintext, and any sufficiently privileged actor on the host can potentially inspect memory or interfere with execution. In AI environments, that creates a problem for both customer data and model IP. The issue is not that encryption fails, but that classic encryption stops being the primary control once computation begins. See the Confidential Computing Guide.

Practical implication: Treat runtime exposure as a first-class control problem, not a storage problem.

How trusted execution environments change the trust model

Trusted execution environments, or TEEs, are hardware-enforced enclaves that isolate code and data from the host operating system, hypervisor, and even privileged administrators. The key shift is that policy is no longer only software-mediated. The processor itself enforces what can be seen while the workload runs, which narrows the opportunities for insider access and infrastructure abuse. For AI, that matters because inference often runs where the operator does not fully control the surrounding platform.

Practical implication: Anchor sensitive inference to hardware isolation, not to administrative trust.

Why attestation and key release are the enforcement point

Remote attestation turns the runtime into something that can be verified before secrets are released. The system checks that the hardware is authentic, the environment is unmodified, and the workload matches expectation. Only then are encryption keys made available. That means trust is established at launch time and renewed through cryptographic verification rather than assumed because a platform says it is secure. If the environment fails verification, the workload does not receive the material it needs to run.

Practical implication: Gate model and data access on attestation, not on network placement or operator assurances.

NHI Mgmt Group analysis

Confidential AI is best understood as a runtime governance control, not a compliance accessory. The article’s core point is that the real security break happens when data or model weights are decrypted for computation and the infrastructure layer can still see them. That changes the control objective from protecting static assets to protecting the active trust boundary. For security teams, this is a shift from storage governance to execution governance.

The identity problem in confidential AI is who can be trusted after launch. The article shows that privileged administrators, host operating systems, and infrastructure operators are all inside the exposure path unless the workload is isolated and attested. That makes the runtime identity of the workload, not just the user identity around it, central to governance. Agentic AI security policy template is relevant here because agentic systems only become governable when access, oversight, and retirement are explicit.

In-use protection is now the deciding control for regulated AI adoption. Confidential AI does not solve data governance by itself, but it removes the architectural reason many programmes cannot run sensitive AI at all. The article is effectively arguing that regulated deployment fails when security is bolted on after architecture decisions are made. Practitioners should treat confidential computing as part of the AI control plane, not as an optional hardening layer.

Hardware-backed attestation is the named trust anchor that makes confidential AI operational. The environment is only usable when the platform can prove what it is and refuse keys when it cannot. That reduces reliance on operator assurances and narrows the space where insider access can matter. For AI programmes handling sensitive records or proprietary models, the practical conclusion is simple: no attestation, no trusted execution.

From our research library:

• Companies using AI governance put over 12 times more AI projects into production, according to a Databricks report.
• Read next: Enterprise AI Copilot Security Guide

What this signals

In-use protection is becoming the control that separates AI experimentation from deployable AI. Organisations can often secure data at rest and in transit, but that does not answer the harder question of who can see the workload while it is running. In regulated AI, the trust decision moves from perimeter controls to runtime assurance.

Runtime trust will increasingly shape AI architecture choices. Teams that plan for confidential computing, attestation, and key release now will have a clearer path to sensitive model deployment later. Those that defer it will keep finding that governance, not model quality, is the rate-limiting factor.

Agentic AI makes the runtime boundary harder to ignore. Once agents start using tools, maintaining context, and touching internal systems, the question is not only whether the model is accurate but whether the environment running it is verifiable. That is why AI governance and identity governance are converging around runtime control.

For practitioners

• Define in-use data protection requirements Classify which AI workloads handle sensitive records, proprietary models, or regulated data and require runtime protection before deployment decisions are made.
• Require attestation before key release Make cryptographic attestation a prerequisite for decrypting data or loading model weights into production inference environments.
• Extend trust checks to GPU execution paths Validate that the protection model covers the GPU pipeline, not only the CPU enclave, when the workload is inference-heavy.
• Separate operator access from workload trust Limit what infrastructure administrators can see or influence once the workload is running, and treat that separation as a design requirement.
• Update AI governance for agentic workflows Account for multi-step agent workflows, tool use, and continuous access to internal systems when setting security requirements for production AI. See the Agentic AI Security Policy Template.

Key takeaways

• Confidential AI addresses the security gap created when sensitive data and model weights must be decrypted during active computation.
• The main risk is not just external attack but privileged access inside the runtime, where host systems and administrators may be able to inspect workloads.
• Attestation and hardware isolation change AI governance from trust-by-policy to trust-by-verification, which is why they matter for regulated deployments.

Related NHI Mgmt Group resources: Confidential Computing Guide · Enterprise AI Copilot Security Guide · Agentic AI Security Policy Template

Key terms

• Confidential AI: Confidential AI is the use of hardware and software controls to protect model data, prompts, and outputs while they are being processed. It reduces exposure to cloud operators, co-tenants, and attackers with infrastructure access. Defenders check memory isolation, encrypted processing paths, access controls, and logging boundaries.
• Trusted Execution Environment: A Trusted Execution Environment is a protected hardware-backed area where code runs with stronger isolation from the host system. In AI inference, the point is not just secrecy in transit, but limiting what operators, hypervisors, and surrounding infrastructure can observe during processing.
• Remote Attestation: Remote attestation is the process of proving, with cryptographic evidence, that a workload is running inside a specific trusted environment. It matters in AI governance because it replaces a verbal trust claim with checkable proof that the inference session used the intended protected boundary.
• In-Use Data Exposure: In-use data exposure is the risk that plaintext information becomes visible while a system is actively processing it, even if it is encrypted at rest and in transit. For AI workloads, this is the gap that makes runtime protection essential when handling sensitive records or proprietary models.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management for practitioners building secure identity foundations. It helps security teams connect identity control to the broader programmes that AI and cloud deployments depend on.