Regulated enterprise AI still runs ahead of trust controls

At a glance

What this is: Fortanix’s survey shows a trust gap in regulated enterprise AI, where leaders are worried about data exposure during processing but still use public cloud AI services and cannot always verify protections.

Why it matters: This matters because IAM, data security and AI governance teams need evidence that sensitive data, model access and runtime controls are enforceable before AI workloads move into production.

By the numbers:

• 93% of AI leaders are concerned about exposing proprietary prompts, models or intellectual property to AI infrastructure providers.
• 61% of organisations send sensitive data to public cloud AI services anyway.
• Only 50% of AI leaders can fully verify that sensitive data stays protected while a model is processing it.
• In the past 12 months, 59% of organisations delayed an AI project because of data security or privacy.

Context

Regulated enterprise AI now exposes a basic governance problem: teams are being asked to place sensitive data into infrastructure they cannot fully validate. The issue is not lack of awareness, but lack of proof that controls remain effective while data is being processed in the AI runtime.

That gap becomes more serious when AI touches customer data, financial data and proprietary prompts. Traditional encryption protects data at rest and in transit, but not during inference, which is the point where AI systems need the data most.

Key questions

Q: How should organisations govern access to data used by AI systems?

A: Treat AI data access as an identity governance problem, not just a data storage problem. Define who or what can use each dataset, what purpose is allowed, and what runtime restrictions apply. Then review humans, service accounts, and AI agents separately so entitlement scope matches actual behaviour rather than a generic AI policy.

Q: Why is encryption not enough for AI systems processing sensitive data?

A: Encryption protects data at rest and in transit, but AI must decrypt and process data to generate outputs. That creates an inference-time exposure window where the runtime, not the storage layer, becomes the main control boundary. Without attestation or trusted execution, organisations cannot prove the data stayed protected while it was being used.

Q: What are the signs that AI security controls are failing in production?

A: Common warning signs include unapproved model behavior, unexpected data access, prompt leakage, suspicious outbound calls, and runtime actions that do not match the workload’s intended function. If teams also see weak visibility into pipelines, inconsistent policy enforcement, or repeated attempts to access sensitive datasets, the control layer is not keeping pace with the AI threat surface.

Q: Should organisations use public cloud AI services for sensitive workloads?

A: Yes, but only when they can prove the processing environment is trustworthy and the data scope is tightly controlled. For sensitive workloads, the decision should depend on whether the organisation can enforce attestation, key-release gating and input minimisation. If not, the safer choice is to limit the workload or keep it out of that service.

Technical breakdown

Why inference-time protection is the hard problem

AI inference creates a different exposure pattern from storage or transport because the model must see usable data to produce an output. That means encryption alone is not enough once data enters the processing environment. In regulated settings, the control question shifts from whether data is encrypted to whether the runtime can be trusted to process it without leakage, tampering or policy drift. Confidential computing, TEEs and attestation are all attempts to make that runtime trustworthy, but the control only works when the environment state can be verified before keys are released.

Practical implication: Treat inference as the control boundary and require runtime verification before sensitive data or keys are allowed into the workload.

Why attestation has become a governance control, not just a technical check

Attestation is the mechanism that proves the processing environment matches an expected state before secrets are released. In AI environments, that state includes CPU, GPU, firmware, drivers and the software stack, all of which can change independently. Without attestation, security teams are relying on claims about the environment rather than evidence. Composite attestation matters because AI workloads are layered systems, and a failure in one layer can invalidate trust in the whole pipeline. This turns attestation into a release gate for sensitive data, not a one-time compliance exercise. See the AI Infrastructure Workload Identity Guide.

Practical implication: Define attestation as a precondition for model access, key release and production deployment.

How key release links identity to trusted execution

Key release is where identity governance meets AI runtime security. If an environment cannot prove its trust state, decryption keys should not be released, even if the workload is otherwise authenticated. That separates authorised access from trusted processing. In practice, this prevents a valid workload credential from becoming a blanket entitlement to decrypt protected datasets or model weights. It also means the identity of the workload, the integrity of the runtime and the condition of the infrastructure must all be evaluated together before access is granted. Read Zero Trust for AI Agents.

Practical implication: Bind secrets release to workload integrity checks instead of treating workload authentication as sufficient.

Threat narrative

Attacker objective: The objective is to obtain sensitive prompts, models or data while the AI system is processing them, or to exploit the organisation’s inability to prove that protection exists.

1. Entry occurs when regulated organisations place sensitive prompts, datasets or model weights into public cloud AI services they cannot fully trust.
2. Escalation happens inside the AI runtime, where data must be exposed for inference and can no longer rely on storage or transport encryption alone.
3. Impact follows when organisations cannot prove protection during processing, leading to delayed deployments, blocked projects and residual exposure of proprietary information.

Breaches seen in the wild

• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Trust in enterprise AI has become a verification problem, not an awareness problem. The article shows that most leaders already understand the risk of exposing sensitive data during AI processing, but many still cannot prove that controls work at inference time. That creates a governance gap between policy intent and runtime assurance, especially in regulated sectors. Practitioners should treat proof of protection as a required control outcome, not an aspirational target.

Inference-time exposure is the named concept this market still underestimates. Data in transit and data at rest are familiar control zones, but inference-time exposure is where AI systems must briefly reveal usable data to compute an output. That means the security question is no longer whether data was encrypted somewhere else, but whether the runtime can be trusted long enough to process it safely. Zero Trust for AI Agents is relevant here because trust must be evaluated at action time, not assumed from access alone.

Composite attestation is becoming the minimum credible control pattern for AI infrastructure. The article’s emphasis on CPU, GPU, firmware and software stack verification reflects how AI trust depends on multiple layers being intact at once. A single-layer view is not enough when the runtime is distributed across hardware and software vendors. Practitioners should expect AI governance to merge with workload integrity, because a trusted model is only as defensible as the environment that executes it.

Regulated industries are now splitting into two camps: those who can prove trust and those who delay deployment. The article shows that compliance friction is not abstract, it is already causing delayed or stopped AI projects. That tells us the competitive issue is no longer who has access to AI, but who can operationalise evidence-based trust without weakening governance. Teams should prepare for AI rollout decisions to hinge on auditable runtime assurance rather than enthusiasm for deployment speed.

From our research library:

• 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, according to the State of Secrets in AppSec.
• Enterprise AI use rose from 55% of organisations in 2023 to 88% in 2025, according to McKinsey’s Global Surveys on the State of AI.
• Read next: Zero Trust for AI Agents

What this signals

Inference-time exposure is becoming the decisive AI security concept. Regulated organisations already know that stored and transmitted data can be protected, but the real governance gap appears when the model must process sensitive inputs. That is where trust has to be proven, not assumed, and where identity, workload integrity and data controls converge.

A practical AI programme will increasingly need a control that behaves like a trust gate rather than a policy label. The combination of attestation, key release and data minimisation is likely to become the default pattern for organisations that cannot afford uncontrolled exposure of prompts, models or regulated records.

AI governance and data security teams should expect more scrutiny over whether protective controls can be demonstrated under load, not just described in design documents. 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, according to the State of Secrets in AppSec.

For practitioners

• Define inference-time trust as a gating control Require proof that the AI runtime is trustworthy before sensitive data, prompts or model weights are allowed into production inference paths.
• Bind key release to environment attestation Do not release decryption keys unless the workload, firmware and software stack all match the approved trusted state for that AI deployment.
• Limit sensitive inputs before model processing Classify and reduce customer data, financial data and proprietary prompts before they reach the AI service so the runtime sees less exposed information.
• Use masking and tokenisation where inference allows it Apply masking or tokenisation to reduce direct exposure of regulated data before it enters the model, especially in shared or third-party environments.

Key takeaways

• Regulated enterprise AI is creating a trust gap because organisations can use public cloud AI services without being able to prove that sensitive data remains protected during processing.
• The article’s numbers show a clear mismatch between concern and assurance, with most leaders worried about exposure but only half able to fully verify runtime protection.
• The most relevant control shift is from policy-based approval to evidence-based runtime assurance, especially where inference, attestation and key release intersect.

Related NHI Mgmt Group resources: Zero Trust for AI Agents · AI Infrastructure Workload Identity Guide · Data Security Guide · AI Governance Guide

Key terms

• Inference-time exposure: The point at which an AI system must process data in usable form to generate an output, creating a temporary but real exposure window. This is the stage where encryption at rest or in transit no longer protects the data, so runtime assurance becomes the critical control.
• Composite Attestation: Attestation that combines evidence from more than one component, such as CPU and GPU layers, into a single trust decision. It is useful for AI workloads that span heterogeneous hardware and need end-to-end integrity checking.
• Trusted Execution Environment: A Trusted Execution Environment is a protected hardware-backed area where code runs with stronger isolation from the host system. In AI inference, the point is not just secrecy in transit, but limiting what operators, hypervisors, and surrounding infrastructure can observe during processing.
• Key release gating: A control pattern that withholds decryption keys until a workload proves it is running in an approved and trusted state. For AI systems, this links access to cryptographic material with runtime integrity, so authentication alone does not grant the ability to process sensitive data.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security and workload identity. It helps security and identity practitioners connect runtime trust to broader identity governance.