Confidential AI and GPU Enclaves Guide: Protecting AI Data in Use

AI workloads decrypt their most valuable assets, model weights, prompts and training data, in GPU and host memory, where cloud operators, hypervisors and host administrators can in principle reach them. Encryption at rest and in transit stops at the point of computation. Confidential AI extends the trusted execution environment (TEE) from the CPU to the GPU, so that inference and training run inside an attested boundary and keys are only released to workloads that prove what they are. This guide explains how GPU confidential computing works on current NVIDIA and cloud platforms, what attestation actually proves, how to wire it to key release, and where the protection ends. It is written for security architects, platform engineers and AI infrastructure teams deciding whether, and how, to run sensitive models in confidential mode.

Key takeaways

• Confidential AI protects data in use, not the model's behaviour. It keeps weights, prompts and outputs away from the infrastructure operator; it does nothing about prompt injection, data leakage through outputs or a poisoned container image.
• Attestation is the product, encryption is the side effect. The security value comes from a relying party checking CPU and GPU evidence against policy before releasing a key. Without that gate, a confidential VM is just an encrypted VM.
• The workload's identity is its measurement. A key broker that releases a model decryption key on a matching measurement is issuing a credential to a non-human identity, and it needs the same ownership, rotation and review as any other.
• The GPU path has a cost. NVIDIA states compute and HBM bandwidth are unchanged in CC-On mode, but CPU to GPU transfers go through encrypted bounce buffers, so data-heavy pipelines slow down.
• Physical attacks are out of scope. The 2025 TEE.fail research showed DDR5 bus interposition against Intel TDX and AMD SEV-SNP; vendors treat it as outside their threat model, so physical security of the host still matters.

What confidential AI is and what it changes

The Confidential Computing Consortium (CCC), a Linux Foundation project, defines confidential computing as protecting data in use by performing computation in a hardware-based, attested trusted execution environment. Confidential AI applies that model to AI pipelines, which previously stopped at the CPU: a GPU outside the TEE received data over PCIe in the clear. GPUs with their own confidential computing mode, starting with the NVIDIA H100, close that gap.

Three questions decide whether it is worth the effort:

• Who are you protecting against? The design target is the infrastructure operator: the cloud provider, a colocation host, a hypervisor or a Kubernetes administrator. NVIDIA's own overview lists protecting proprietary model weights from infrastructure operators, protecting regulated data during processing, and sovereign AI deployments as the primary use cases.
• Who owns which asset? NVIDIA is explicit that the trust boundaries do not automatically protect end-user data from the model owner. If a user sends a prompt to a provider's model in a TEE, the provider still controls the code that sees the prompt unless the provider publishes and lets users verify that code.
• What do you do with the attestation? If nothing consumes the evidence, nothing changes. The control that matters is a policy that refuses to release a key, model or dataset to an unverified environment.

Confidential AI sits alongside, not in place of, other privacy techniques. The OWASP Top 10 for LLM Applications 2025 lists federated learning, differential privacy and homomorphic encryption as mitigations for LLM02 Sensitive Information Disclosure. Those techniques change what data the model sees; a TEE changes who else can see it while it is processed. See the Confidential Computing Guide for CPU enclave basics.

How GPU confidential computing works

NVIDIA's H100 confidential computing design, described in its developer documentation, pairs a CPU confidential VM with a GPU that has its own hardware root of trust.

• Root of trust and boot: an on-die hardware root of trust anchors a secure, measured boot. At boot the GPU firmware scrubs GPU state and memory and configures a hardware firewall before PCIe is enabled.
• Device identity: each GPU has a unique ECC-384 identity key fused into the chip. The device certificate can be read with nvidia-smi and verified against the NVIDIA certificate authority, with revocation checked through the NVIDIA OCSP service.
• Secure session: the driver inside the CPU TEE establishes an SPDM session with the GPU. Command buffers and CUDA kernels are encrypted and signed before crossing PCIe.
• Bounce buffers: the GPU cannot read the CPU TEE's private memory, so the driver encrypts data into a bounce buffer in shared system memory and the GPU decrypts it inside its protected region. Unified Memory pages are encrypted when migrated.
• HBM: NVIDIA states GPU high-bandwidth memory is not encrypted; it is treated as inside the boundary and resistant to common physical attack tools.

GPU CC modes

The host sets the mode through in-band PCIe commands or out-of-band BMC commands. That is why the mode must be an attested claim checked by policy, not an assumption: a GPU left in DevTools mode looks like a working GPU to the workload.

Deployment modalities

NVIDIA notes that none of these modalities eliminates shared-hardware, fabric, physical or side-channel risk, and directs readers to model-specific threat documentation before releasing secrets.

Platform options

Supported configurations change quickly, so treat this table as a map of where to look, and confirm the current matrix before procurement.

Multi-GPU training adds the interconnect to the boundary. NVIDIA ships a Protected PCIe (PPCIe) SDK for multi-GPU confidential configurations; check the compatibility matrix for which HGX and NVLink topologies are supported in confidential mode before designing a training cluster around it.

Attestation and key release

Attestation is where confidential AI becomes an identity problem. The workload proves what it is with hardware-signed evidence, a verifier appraises that evidence against reference values, and a key broker issues a secret. NVIDIA's overview states the point directly: attestation supplies evidence for a trust decision but does not decide on its own whether a workload gets secrets.

1. Collect CPU evidence: an SEV-SNP or TDX report from the confidential VM, binding the guest's launch measurement and a nonce.
2. Collect GPU evidence: the GPU's signed attestation report and certificate chain, retrieved by the driver.
3. Verify the GPU: NVIDIA Remote Attestation Service (NRAS) is the primary verifier; it uses the Reference Integrity Manifest (RIM) service for expected firmware values and OCSP for revocation. A local verifier exists for air-gapped sites, but NVIDIA warns its revocation data and reference values can go stale.
4. Appraise against policy: CC mode is On, firmware and driver versions meet the minimum, the CPU TEE type is the one expected, the guest image measurement matches a release you built.
5. Release the secret: only then does the key broker return the key that decrypts the model weights, dataset or TLS private key.

Key broker patterns

• Azure Secure Key Release: an exportable HSM-backed key in Key Vault Premium or Managed HSM carries a release policy. The policy names the MAA authority and claims such as x-ms-isolation-tee.x-ms-attestation-type equals sevsnpvm and x-ms-isolation-tee.x-ms-compliance-status equals azure-compliant-cvm. Microsoft notes SKR applies to keys only, not to secrets or certificates, and that SKR will also release to non-confidential Trusted Launch VMs if the policy allows, so the policy must require the confidential claims.
• Trustee (Confidential Containers project): the Key Broker Service (KBS), Attestation Service (AS) and Reference Value Provider Service (RVPS) run outside the pod; the Attestation Agent and Confidential Data Hub run inside the guest. The project advises deploying Trustee itself in a trusted environment, because it decides which guests get secrets.
• AWS KMS with Nitro Enclaves: key policies condition on enclave measurements. AWS recommends PCR3 (parent instance IAM role) with PCR8 (image signing certificate) for flexibility. Enclaves in debug mode produce all-zero PCRs and cannot pass cryptographic attestation.

Treat every released key as an NHI credential: it needs an owner, a rotation schedule, a scope limited to one model or dataset, and logs of every release. See the AI Infrastructure Workload Identity Guide and the Secrets Management Guide.

Threat model: what is and is not covered

Mapping to adversary frameworks: MITRE ATLAS AML.T0025 Exfiltration via Cyber Means (stealing weights from storage or memory) is the technique confidential AI targets; AML.T0024 Exfiltration via AI Inference API is not affected because it uses the legitimate interface; AML.T0010 AI Supply Chain Compromise passes straight through attestation if the poisoned component is in the measured image. In ATT&CK terms, CC raises the bar against T1005 Data from Local System and T1552.004 Private Keys by a privileged host user, not by code running in the guest.

Decision guidance

• Choose confidential VMs when you run a small number of large inference or fine-tuning servers and want the fewest moving parts. Most workloads need no application code change.
• Choose confidential containers when a platform team serves many models to many internal tenants and already runs Kubernetes; budget for running Trustee or an equivalent broker and for workload policy enforcement.
• Choose confidential bare metal when you own GPUs placed with a third party and want the facility operator outside the boundary, while accepting that your own administrators remain inside it.
• Skip GPU CC for public or low-sensitivity models where the operator is already trusted under contract, or for data-movement-heavy jobs where the bounce-buffer overhead (NVIDIA cites CPU encryption limits of roughly 4 GB/s on the CPU to GPU path) outweighs the risk reduction.
• Multi-party AI (two organisations pooling data to train or score a model neither may see) is a good fit for confidential computing with jointly agreed workload measurements, as Google's Confidential Space is designed for.

Small vs large organisations: a small team should consume a managed confidential GPU instance and the cloud's native attestation and key release rather than operate NRAS integration and a key broker itself. A large organisation or AI provider should run its own relying-party policy, publish reference values per release, and consider a transparency model like Apple's Private Cloud Compute, where devices only send requests to nodes whose attested measurements match software published in an append-only log, and the system has no privileged runtime access such as remote shells.

How to implement

1. Classify assets and adversaries: list which models, datasets and prompt streams justify operator-proof protection, and write down the adversary (cloud operator, host admin, co-tenant). Stop if the answer is "nobody we do not already trust".
2. Confirm the platform matrix: GPU SKU, CPU TEE, driver, VBIOS and guest image versions on the vendor's current compatibility list, plus production licensing terms.
3. Build a reproducible image: pinned dependencies, signed build provenance, no debug tooling, no SSH. Record its measurement as a reference value.
4. Stand up attestation in audit mode: collect and verify CPU and GPU evidence for every launch and log the result, but still release keys through the old path. Fix failures (mode not On, stale firmware, unknown measurement) before enforcing.
5. Encrypt the asset and move the key behind policy: encrypt model weights at rest with a key that only the broker can release; set the release policy to require confidential claims, CC-On and the reference measurement.
6. Pilot one model: run a single inference service in enforcing mode, measure throughput and latency against a CC-Off baseline, and test the failure path (deliberately launch an unapproved image and confirm no key is released).
7. Roll out by sensitivity tier, with a runbook for updating reference values on every image release, driver update and firmware update.
8. Rollback: keep the previous image measurement in policy for a short overlap window and remove it after cut-over; for an emergency, revert to the earlier measured image, never by loosening the policy to accept any measurement.

Detection and measurement

Log every attestation verdict, every key release and every policy change, and send them to the SOC alongside cloud control-plane logs.

• Alerts: attestation failures; any key release to a non-confidential claim set; GPU reported in DevTools or Off mode on a production host; changes to release policies or reference values outside the release process; use of a local verifier where NRAS is expected.
• Revocation: track NVIDIA OCSP and firmware advisories and fail closed when a device certificate is revoked.

Common failure patterns

• Encryption without a gate: the team turns on a confidential VM but the model key is still fetched with a normal managed identity. The host operator cannot read memory, but anyone with that identity can download the key and the weights elsewhere.
• Policy that accepts too much: an Azure SKR policy checking only that a token was issued, not the SEV-SNP and compliance claims, releases to ordinary Trusted Launch VMs.
• Poisoned code inside the boundary: attestation proves the image is the one you built, not that it is safe. The xinference PyPI compromise put a credential stealer inside an AI inference server package that ran on import; baked into a measured image, it would attest perfectly and read every secret the TEE released.
• Debug modes in production: GPUs left in CC-DevTools, or enclaves in debug mode, behave normally but provide no protection. Only policy checks catch them.
• Assuming physical safety: TEE.fail showed a sub-$1,000 DDR5 interposer could extract attestation keys from Intel TDX and SGX platforms and signing keys from SEV-SNP guests, and that forged CPU attestation could be used to relay GPU attestation from other machines. Intel and AMD consider interposers out of their threat model, so siting matters.
• Stale reference values: a driver update breaks attestation, and the quick fix of accepting any measurement quietly turns the control off.

Framework mapping

NIST IR 8320 on hardware-enabled security sets the wider context: securing the platform on which workloads run is the first layer that lets higher-layer controls be trusted.

Practitioner checklist

• Write down the adversary confidential AI is meant to stop, per model or dataset.
• Confirm GPU, CPU TEE, driver and image versions against the vendor's current compatibility matrix.
• Encrypt model weights at rest with keys that only an attestation-gated broker can release.
• Require CC-On mode, CPU TEE type and the approved image measurement in every release policy.
• Verify GPU evidence through NRAS with OCSP revocation, or document why a local verifier is used.
• Build images reproducibly with pinned, provenance-checked dependencies and no debug access.
• Run attestation in audit mode first, then enforce on one pilot model with a tested failure path.
• Assign an owner to every release policy and broker key, and review them quarterly.
• Alert on attestation failures, policy edits and any GPU in DevTools mode.
• Keep physically sensitive workloads in facilities whose operators and access you trust.

Standards and references

• Confidential Computing Consortium: About the Consortium
• NVIDIA: Confidential Computing Overview
• NVIDIA: Attestation Suite Documentation
• Microsoft Learn: Secure Key Release with Azure Key Vault and Azure Confidential Computing
• Google Cloud: Confidential VM overview
• Confidential Containers: Attestation (Trustee)
• NIST: IR 8320 Hardware-Enabled Security
• TEE.fail: Breaking Trusted Execution Environments via DDR5 Memory Bus Interposition

Related NHI Mgmt Group resources: Confidential Computing Guide · AI Infrastructure Workload Identity Guide · AI Model Security Guide · Secrets Management Guide