Securing generative AI workloads is now an identity problem

At a glance

What this is: This is a vendor research report on why securing generative AI workloads is becoming harder, with 41% of organisations reporting security risks and vulnerabilities during GenAI integration.

Why it matters: It matters because GenAI security now intersects with NHI, autonomous system access, and human governance decisions about data exposure, workload trust, and operational continuity.

By the numbers:

• 41% of organisations report struggling with security risks and vulnerabilities when integrating GenAI into their AI infrastructure.

👉 Read Cyera's report on securing workloads for generative AI

Context

GenAI workload security is the problem space here, and the identity question sits underneath it. When GenAI systems process confidential data, interact with tools, or support operational workflows, the security model has to cover both access and data handling, not just model behaviour.

The report frames the risk as a combination of sensitive data exposure, compliance pressure, IP protection, and business continuity. For IAM and security teams, that means GenAI cannot be governed as a simple application layer. It behaves more like a workload with identity, privilege, and data access boundaries that must be explicit and reviewable. For deeper background on workload identity patterns, see the Guide to SPIFFE and SPIRE.

Key questions

Q: How should security teams govern GenAI workloads that access sensitive data?

A: Treat the workload as a governed identity with explicit scope, named ownership, and continuous review. The key is to map which data sources, tools, and output paths the GenAI system can reach, then limit those permissions to the smallest task boundary possible. Without that discipline, the workload becomes a broad access path instead of a controlled service.

Q: Why do GenAI integrations create security risk even when the model is approved?

A: Approval of the model does not guarantee control over the runtime environment. Risk appears when the workload gains new connectors, expands its data sources, or inherits broader permissions from adjacent systems. A secure approval process has to cover the full access chain, not just the model artefact.

Q: What do IAM and security teams get wrong about GenAI access control?

A: They often focus on the model or the application wrapper and ignore the data paths behind it. In practice, the main exposure comes from what the workload can read, cache, forward, or trigger across connected systems. That is why entitlement review has to include the runtime ecosystem around the model.

Q: How can organisations tell whether GenAI security controls are actually working?

A: Look for evidence that access scope, connector trust, and data handling are all reviewable in the same control process. If teams cannot show who owns the workload, what it can access, and when those permissions were last reassessed, the control is not functioning as a governance mechanism.

Technical breakdown

GenAI workload identity and privilege boundaries

A GenAI workload is not just a model endpoint. It is a runtime identity that may call data sources, invoke tools, and read or write sensitive information during execution. The security issue is not whether the model is clever, but whether its identity is bounded tightly enough for the data and actions it can touch. In workload terms, privilege should map to the task, the dataset, and the execution context. If those boundaries are loose, the model becomes an access pathway rather than a controlled service.

Practical implication: treat GenAI workloads as governed identities with explicit scopes, not as generic application traffic.

Sensitive data exposure in AI pipelines

GenAI systems often ingest prompts, documents, logs, and embedded context from multiple sources. That makes them a data concentration point, which is why discovery and classification matter as much as access control. If the pipeline is not segmented, sensitive information can bleed across prompts, caches, vector stores, and downstream tools. The risk is not only exfiltration. It is also unintended retention, over-sharing, and policy drift across the full workflow.

Practical implication: map where sensitive data enters, persists, and leaves the GenAI workflow before expanding usage.

Why workload trust needs continuous verification

GenAI security fails when trust is assumed at provisioning time and never revisited. Workloads change, data changes, connectors change, and the identity path can widen as teams add integrations. That is why zero trust thinking matters here. The question is not whether the workload was approved once. The question is whether it remains bounded after configuration drift, connector sprawl, and changes in the surrounding control plane.

Practical implication: require continuous verification for GenAI workloads as integrations and entitlements change.

Breaches seen in the wild

• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.
• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

GenAI workload security is becoming an identity and data governance problem, not just a model protection problem. Once a generative system can consume sensitive inputs and trigger downstream actions, the control surface moves from the model to the workload identity. That means the security conversation has to include access scope, connector trust, and data handling boundaries. Practitioners should assess GenAI through the same governance lens used for other high-privilege workloads.

Runtime data access changes the risk profile faster than traditional review cycles can track. GenAI systems often pull from multiple stores, caches, and tool integrations in a single session. That creates a moving target for entitlement review and data classification. The practical implication is that identity governance for GenAI has to account for dynamic data paths, not just static service permissions.

Secure GenAI controls are only credible when they align with workload identity patterns already used in modern zero-trust architectures. The closest fit is an architecture that treats every workload as a distinct identity with explicit verification, scoped access, and observable trust relationships. Teams that already understand SPIFFE-style workload identity are better positioned to manage GenAI access paths without relying on broad perimeter assumptions. Practitioners should align GenAI governance with workload identity standards rather than ad hoc application exceptions.

Ephemeral workload access exposes a recurring governance gap: visibility is often weaker than privilege. Security teams may know a GenAI system exists, yet still lack reliable control over what it can reach, store, or forward. That gap is why many programmes struggle when AI is introduced through existing cloud and data platforms. Practitioners should close the visibility gap before scaling usage across more sensitive workflows.

From our research:

• Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared to nearly 1 in 4 for securing human identities, according to The State of Non-Human Identity Security.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap, according to The State of Secrets in AppSec.
• If you are mapping GenAI access paths, start with Guide to SPIFFE and SPIRE for workload identity patterns that fit continuous verification.

What this signals

Workload identity is becoming the organising principle for GenAI governance. As GenAI systems move closer to production data and action paths, the old separation between application security and IAM stops holding. Teams need to decide whether the workload is treated like a tightly scoped service identity or a loosely managed integration, because that choice determines how far the blast radius can extend.

The most practical signal for programme maturity is whether security teams can describe every GenAI connector, every data source, and every privilege path without cross-functional guesswork. If they cannot, the organisation has visibility debt, and that debt will surface first in audit, incident response, or compliance reviews.

With 85% of organisations lacking full visibility into third-party vendors connected via OAuth apps, according to The State of Non-Human Identity Security, the same blind spot is likely to show up when GenAI systems are layered onto existing SaaS and cloud estates. The governance response is to treat AI access as part of the broader identity control plane, not as an isolated innovation workstream.

For practitioners

• Classify each GenAI workload as a governed identity Inventory the model, orchestration layer, connectors, and downstream data paths as one access chain. Assign explicit ownership, scope, and review responsibility for the full runtime path.
• Map sensitive data paths through prompts and connectors Trace where confidential data enters, is transformed, cached, exported, or logged. Include vector stores, retrieval layers, and integration points in the same data-flow review.
• Tighten workload access to task-scoped privileges Remove broad read or write permissions that are not required for the current use case. Separate production GenAI workloads from adjacent service accounts and human-administered tooling.
• Verify ongoing trust instead of one-time approval Reassess connector access, data sources, and execution scope whenever the GenAI workflow changes. Treat configuration drift as a security event, not a routine admin update.

Key takeaways

• GenAI workloads are now identity-bearing systems that need explicit access boundaries, not just model-level oversight.
• The main risk is not the model alone, but the connected data paths, connectors, and permissions surrounding it.
• Teams that align GenAI governance with workload identity and continuous verification will have a much clearer control model than those relying on one-time approval.

Key terms

• GenAI Workload Identity: The identity assigned to a generative AI workload so it can be governed like any other non-human system. It covers the runtime permissions, connectors, and data paths the workload can touch, which is essential when the system reads sensitive data or triggers actions across other services.
• Connector Trust: The confidence an organisation places in the integrations that let a GenAI system reach external data or tools. In practice, connector trust is a security boundary, because each additional connection expands the workload's effective privilege and can introduce new exposure paths if not reviewed.
• Data Path Governance: The discipline of tracking where sensitive data enters, moves through, and leaves a GenAI workflow. It includes prompts, caches, vector stores, logs, and downstream outputs, because each stage can create retention, disclosure, or policy drift if the control model is incomplete.

Deepen your knowledge

GenAI workload identity and privilege boundaries are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are extending identity governance into AI-driven workloads, it is worth exploring.

This post draws on content published by Cyera: Importance of Securing Workloads for Generative AI Report AI. Read the original.