Enterprise AI security risks expose identity and data control gaps

At a glance

What this is: Orca Security maps enterprise AI security risk to recurring gaps in testing, explainability, data handling, adversarial abuse, supply chain, and shadow AI.

Why it matters: This matters because AI services inherit identity and data exposure from the cloud estate, so IAM, DSPM, and governance controls must extend into AI pipelines, not sit beside them.

Context

Enterprise AI security is not just a model issue. The practical failure mode is that AI features sit on ordinary cloud accounts, inherit existing identity sprawl, and create new paths for data exposure, tool abuse, and uncontrolled outputs.

For IAM and security teams, the key question is whether AI services are governed as first-class workloads with owners, access boundaries, and data controls. When they are treated as add-ons to existing cloud programmes, the gaps appear in testing, explainability, and third-party dependency management.

Key questions

Q: What breaks when identity governance is discussed without cloud and AI context?

A: Identity governance becomes incomplete when it ignores how access is actually provisioned and used in cloud and AI systems. Teams can end up with policies that look sound on paper but do not address ephemeral workloads, service accounts, agentic access, or third-party integrations. That gap creates blind spots in approvals, monitoring, and offboarding, which is where risk accumulates.

Q: Why do AI systems create more data exposure risk than human users with the same access?

A: AI systems can process and combine information at machine speed without the judgment humans use to ignore irrelevant or sensitive material. That means a broad permission set becomes far more dangerous when the actor is a model-driven workflow. The risk is not just access, but automatic discovery and reuse of everything within reach.

Q: What do security teams get wrong about AI safety testing?

A: The common mistake is treating AI safety testing as if it were just another security scan. It is not. Safety testing is about proving how a model or agent fails under pressure, while traditional security tooling is about who can access the system. Those are different governance questions and need different evidence.

Q: How should organisations govern shadow AI without blocking legitimate use?

A: Start with approved-use policy, tool inventory, and data classification. Then require that any AI system handling internal information has named owners, logged access, and defined credential paths. The goal is not prohibition, but visibility and control. If a tool cannot be inventoried or monitored, it should not process sensitive data.

Technical breakdown

Why AI workloads inherit cloud identity risk

Most enterprise AI features run on standard cloud infrastructure, so they inherit the same IAM roles, service accounts, API scopes, and data paths as other workloads. The problem is not the model alone, but the surrounding control plane that lets prompts reach data, tools, and external services. When those relationships are not inventoried, a chatbot, copilot, or retrieval pipeline can act with broader access than the business intended. That creates a governance problem, not just a security one. The operational challenge is to map each AI component to an owner, a data class, and an authorization boundary before it is promoted into production.

Practical implication: Treat every AI feature as a governed workload with explicit identity and data boundaries.

How prompt injection and tool abuse turn model access into impact

Prompt injection is an attack pattern where untrusted content influences a model’s instructions, while tool abuse occurs when the model can call APIs or services with ambient permissions. The technical issue is not only bad text output. It is that the model can convert a malicious instruction into a real action if the hosting application exposes tools, retrieval, or internal URLs without strong separation. This becomes more dangerous when the model summarizes email, tickets, or web content, because attackers can hide commands inside normal-looking data. The control problem is upstream content trust and downstream action validation.

Practical implication: Separate untrusted content from instructions and constrain which tools AI systems can invoke.

Why supply chain weakness matters in AI pipelines

AI systems depend on base models, fine-tuning libraries, containers, plugins, datasets, and hosted services. A compromise anywhere in that chain can alter outputs, leak data, or introduce malicious behaviour without looking like a classic software defect. That makes AI supply chain risk broader than standard dependency management because the assets include models, prompts, retrieval content, and plugin ecosystems. Organisations need to know which third-party services can influence inference and which components are capable of changing model behaviour after deployment. The governing issue is provenance across the whole AI stack, not just package integrity.

Practical implication: Track provenance for models, datasets, plugins, and serving components as part of one supply chain.

Threat narrative

Attacker objective: Exploit AI-assisted workflows to reach sensitive data, manipulate outputs, or trigger privileged actions through trusted integrations.

1. Entry occurs when employees introduce shadow AI tools, untrusted content, or third-party model dependencies into business workflows.
2. Credential or data access follows when the AI service inherits cloud permissions, API scopes, and retrieval access beyond the intended task boundary.
3. Escalation happens when prompt injection or tool abuse turns model interaction into real actions against connected systems or data stores.
4. Impact emerges as sensitive data, internal documents, or downstream actions are exposed, altered, or exfiltrated through AI-enabled workflows.

Breaches seen in the wild

• ShadowRay 2024: Attackers took over internet-exposed Ray AI clusters via the disputed CVE-2023-48022, exposing cloud keys, AI API tokens and GPU compute.
• Hugging Face API tokens exposed 2023: Lasso Security found 1,681 live Hugging Face tokens in public code, 655 with write access, reaching 723 organisations including Meta; all were revoked.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI security failures are cloud identity failures until proven otherwise: enterprise AI features inherit the IAM roles, storage paths, and third-party integrations of the environments that host them. That means the operational unit is not the model alone but the whole workload boundary, including credentials, data flows, and ownership. Security teams that separate AI review from cloud identity review will miss the access paths that matter. The practitioner conclusion is simple: AI governance must start with the same identity inventory discipline used for other production services.

Testing gaps are the first governance failure, not a quality issue: security teams often benchmark AI systems for accuracy while under-testing tool calls, tenant isolation, and adversarial inputs. That leaves a false sense of control because the dangerous failures happen in integration paths, not in model scores. The article’s recurring themes show that evaluation has to include abuse cases, not just functional checks. Practitioners should treat AI test coverage as a control objective tied to business impact.

Shadow AI creates unowned identity and data pathways: employees using unsanctioned AI tools create access routes that no inventory, recertification, or data policy can reliably govern. This is not merely shadow IT with a new interface. It is a parallel data-processing path with unclear retention, authentication, and residency assumptions. The named concept here is shadow AI control debt: the longer sanctioned alternatives lag behind user demand, the more unmanaged AI usage accumulates. Leaders need one risk register for these services because ownership without visibility is not governance.

AI supply chain risk extends beyond software dependencies to model provenance: base weights, fine-tuning assets, plugins, and container images all influence behaviour after deployment. That means procurement, security review, and runtime monitoring have to follow the same artifact across its lifecycle. A model update is not just a version change. It is a potential change in data exposure, tool use, or compliance posture. Practitioners should require provenance and ownership for every component that can alter inference outcomes.

Least privilege becomes a pipeline property in AI environments: the practical question is not whether a model can answer correctly, but whether it can reach only the data and tools required for that answer. If the answer depends on broad API scopes or shared retrieval access, the programme has accepted excess authority as a design pattern. That weakens both prevention and incident response because investigators cannot separate intended from unintended access. The implication is that AI governance must be measured through entitlement scope, not only policy statements.

From our research library:

• 92% of organisations expose NHIs to third parties, raising concerns about supply chain security, according to the Ultimate Guide to NHIs.
• Read next: Shadow AI and AI Agent Discovery Guide

What this signals

Shadow AI control debt: when sanctioned AI access is slow or incomplete, unmanaged employee use grows faster than policy can catch up. That leaves data residency, retention, and ownership assumptions outside the control register, which is why discovery and approved alternatives need to be part of the same programme.

AI governance is becoming a workload governance problem. Security teams should expect AI services to appear wherever cloud identity, third-party APIs, and sensitive data already intersect, which makes entitlement scope and data classification the first control questions to answer.

The practical boundary is not whether an AI system can reason well, but whether it can be trusted with the right data and only the right actions. Once tool use, retrieval, and output handling are connected, the control objective shifts to inventory, authorization, and continuous testing.

For practitioners

• Build an AI asset inventory Inventory models, endpoints, datasets, vector stores, plugins, and third-party APIs with owners, business criticality, and last review date.
• Classify AI data paths before production Define which datasets may train, which may be retrieved, which must stay regional, and which sources are prohibited from entering prompts or embeddings.
• Constrain AI tool permissions Limit service account scopes and API access so AI systems can only reach the tools and data needed for the approved task.
• Expand testing beyond model accuracy Add red-team exercises, adversarial prompt suites, and integration tests for authentication, authorization, and tenant isolation.
• Separate sanctioned AI from shadow use Publish approved tools, monitor discovery signals where policy allows, and give employees a logged alternative that reduces the incentive to bypass controls.

Key takeaways

• Enterprise AI risk is driven by the same control failures that already affect cloud workloads, including identity sprawl, weak data boundaries, and unmanaged third-party access.
• The article’s central pattern is operational, not theoretical: AI features create new paths for prompt injection, data exposure, and supply chain compromise when they are deployed without governance.
• Security teams need AI inventory, scoped permissions, and testing that covers adversarial use cases if they want to govern AI as a production workload.

Key terms

• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• AI Asset Inventory: A living register of every AI-related asset in an organisation, including models, agents, datasets, notebooks, endpoints, and embedded AI services. It links technical detail to ownership, data exposure, lifecycle status, and controls so governance can operate on facts rather than assumptions.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• AI Supply Chain Security: AI supply chain security is the practice of protecting the people, data, models, code, tools, and services used to build and run AI systems. It covers risks from training data, model artifacts, dependencies, prompts, connectors, and deployment pipelines, with controls for integrity, provenance, access, and change monitoring across the full lifecycle.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.