AI security tools miss cloud identity risk across ML attack phases

At a glance

What this is: This is an analysis of why AI security tools leave cloud identity and infrastructure exposure uncovered across the ML attack lifecycle.

Why it matters: For IAM, cloud security, and NHI practitioners, it shows why model-centric testing is incomplete unless teams can govern execution roles, storage permissions, and shadow AI in production.

By the numbers:

• Shadow AI deployments affect 40% of enterprise cloud environments, according to the 2024 Gartner AI TRiSM market survey cited by Orca Security.

Context

AI security tools are often evaluated as if model robustness, prompt injection, privacy leakage, and notebook hygiene together close the risk gap. They do not. The cloud identity problem sits underneath those categories: execution roles, storage permissions, inference access, and untracked AI services can still create the path into production even when the model layer is well tested.

The practical issue is that ML attack phases begin in the cloud control plane long before an attacker reaches the model. A security stack that cannot see IAM misconfiguration, exposed notebooks, or shadow AI deployments will miss the attack path that actually connects AI workloads to sensitive data and operational systems. For identity teams, that makes AI security a governance problem as much as a testing problem.

Key questions

Q: What breaks when AI security posture checks are missing from cloud and data platforms?

A: Without posture checks, teams lose visibility into excessive permissions, weak access controls, and risky configurations across AI assets. The usual result is delayed detection, noisy alerts, and blind spots that make investigations harder. In regulated environments, the same gap also slows audit preparation because evidence is scattered across tools instead of linked to the AI workload itself.

Q: Why do AI workloads need IAM and cloud posture controls as well as model testing?

A: Because the model is usually not the first thing an attacker touches. Misconfigured storage, overly broad roles, and exposed services create the access path that makes model-layer risks exploitable. IAM and posture controls decide whether the AI system can be reached at all, which is why they belong in the same governance discussion.

Q: How should security teams measure whether AI is helping rather than hiding risk?

A: Security teams should measure AI using outcome metrics that include access scope, session length, revocation speed, and auditability. Productivity alone can look positive while identity risk grows underneath it. A useful scorecard ties AI output to the controls that bound its privilege and prove who or what acted at runtime.

Q: Should organisations treat shadow AI as a security risk or an innovation issue?

A: Treat it as both, but govern it first as a security risk. Shadow AI becomes dangerous when it can reach data, call APIs, or make decisions outside approved control paths. Security teams should build intake and review processes that allow safe experimentation without leaving identities and permissions unmanaged.

Technical breakdown

ML attack phases begin in cloud identity, not at the model

AI attack coverage is often described in terms of model manipulation, prompt injection, or privacy leakage, but the attack path usually starts earlier. In production, reconnaissance targets data stores and exposed notebooks, initial access arrives through misconfigured storage or an overpermissive execution role, and the attacker then uses that foothold to reach the model or adjacent cloud services. The important distinction is that model security tools inspect behaviour inside the model layer, while cloud identity controls determine whether the workload can be reached in the first place.

Practical implication: Map each AI workload to the identities and permissions that can reach it before treating model testing as sufficient.

Attack path analysis is the missing control plane for AI risk

Individual findings are misleading when they are detached from the permissions chain that makes them exploitable. An exposed training bucket, an overprivileged execution role, and an open inference endpoint may each look moderate on its own, but together they form a critical path from cloud entry to data access and lateral movement. Attack path analysis matters because it ties findings to the execution context that turns weak posture into real exposure, which is the level at which IAM and cloud governance decisions are made.

Practical implication: Correlate IAM roles, storage access, and endpoint exposure before assigning severity or prioritisation.

Shadow AI creates an identity governance blind spot across ML estates

Shadow AI is not only an asset inventory problem. It is an identity and governance failure because untracked services, SDKs, and endpoints are still consuming cloud permissions, credentials, and data paths even when security teams do not know they exist. That means traditional review cycles can miss entire workloads, especially when teams assume that known inventories represent the full AI estate. For NHI governance, the control failure is visibility into who or what is actually authorised to operate in production.

Practical implication: Treat untracked AI services as identity governance exceptions, not just discovery findings.

Threat narrative

Attacker objective: The attacker aims to pivot from an AI workload foothold into cloud data and infrastructure that support production systems.

1. Reconnaissance starts against AI data repositories, notebooks, and exposed cloud services rather than against the model itself.
2. Initial access commonly follows misconfigured storage, exposed notebooks, or overly broad execution roles attached to AI workloads.
3. Escalation occurs when those permissions allow movement from the AI workload into inference endpoints, sensitive data stores, or other cloud resources.
4. Impact is the compromise of AI-adjacent data and infrastructure through a path that model-only tooling never sees.

Breaches seen in the wild

• reviewdog Action compromise 2025: A stolen maintainer token poisoned reviewdog/action-setup, leaking CI secrets including the tj-actions bot token used in the next attack.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Cloud AI security is now an identity governance problem disguised as a model-security problem. The article’s central point is that tool coverage across robustness, prompt safety, and privacy still leaves the cloud control plane ungoverned. That means execution roles, storage permissions, and endpoint exposure remain the decisive risk surface. The practitioner implication is straightforward: if the identity layer is invisible, the AI security programme is incomplete.

Attack path analysis is the right organising concept for AI workloads. A medium or high misconfiguration finding is not the same thing as a true compromise path, and the distinction matters when workloads can chain role permissions, bucket access, and network exposure. This is where cloud governance beats feature counting. Practitioners should prioritise the chain, not the isolated alert.

Shadow AI is a lifecycle failure, not just a discovery gap. The named concept here is cloud identity blind spot, which describes AI services that operate with valid cloud permissions outside formal inventory and review. They remain live, reachable, and able to access data even when no one is governing them as part of the production estate. The implication is that discovery without ownership and offboarding discipline leaves the risk intact.

Model testing does not substitute for cloud access control. Open-source tools can exercise adversarial robustness, prompt injection, and privacy leakage, but those checks do not tell you whether the training job can read a sensitive bucket or whether the inference endpoint is callable from unintended principals. That separation is exactly why AI security and IAM teams need a shared control model. Practitioners should align AI security reviews with entitlement reviews, not treat them as separate disciplines.

The market signal is convergence between AI security and cloud security operations. Tool stacks that stop at model behaviour are increasingly mismatched to how AI systems fail in production. The article implicitly validates a broader shift toward posture management, identity visibility, and attack path prioritisation for AI estates. Security teams should assume that AI governance will be measured by cloud control outcomes, not by the breadth of a model-testing catalogue.

From our research library:

• Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
• Read next: Identity Security Posture Management (ISPM) Guide

What this signals

Cloud AI posture management is becoming the practical control layer for AI estates. Model testing, prompt safety, and privacy tools matter, but they do not answer the governance question that identity teams have to own: who or what can actually reach the workload in production. The control point shifts to execution roles, storage permissions, and endpoint exposure, because those are what turn AI activity into a governed identity problem.

Shadow AI is a programme design flaw when inventories and entitlement reviews are disconnected. If an AI service can be deployed, granted access, and used without being pulled into the normal review cycle, then the organisation has built a parallel identity estate. Practitioners should assume the same lifecycle controls used for service accounts and workload identities must extend to AI services and agents.

Attack path correlation is the difference between noisy findings and actionable risk. When a workload’s IAM role, data access, and runtime exposure are assessed together, security teams can see which weaknesses actually compose into a breach path. That is the level at which AI governance becomes operational rather than theoretical.

For practitioners

• Map AI workload identities to cloud attack paths Trace each training job, notebook, and inference endpoint back to its execution role, storage access, and network reachability so you can see the exploitable path rather than a list of isolated findings.
• Inventory shadow AI as an identity issue Treat untracked AI services, SDKs, and endpoints as governed identities with owners, permissions, and offboarding requirements instead of as generic discovery noise.
• Correlate infrastructure findings before scoring severity Combine IAM execution role scope, bucket permissions, and endpoint exposure into one prioritised risk view before deciding what is medium, high, or critical.
• Embed AI security checks into CI/CD and cloud workflows Run notebook scanning, configuration checks, and cloud posture controls in the delivery pipeline so AI workloads are assessed where they are actually deployed.

Key takeaways

• AI security tools that stop at model and notebook testing leave cloud identity risk open across production AI workloads.
• The article highlights that shadow AI reaches a meaningful share of enterprise cloud environments and can remain completely unscanned by workload-limited tools.
• Practitioners need correlated visibility across IAM, storage, and endpoint exposure if they want to reduce AI risk in production rather than just document it.

Key terms

• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Attack Path Analysis: Attack Path Analysis is the process of mapping how an attacker could move from an initial foothold to a valuable target. It examines identities, permissions, network reachability, misconfigurations, and trust relationships to identify realistic routes of compromise. The goal is to prioritize controls that break the shortest and most likely paths.
• Cloud Security Posture Management: Cloud Security Posture Management is a set of tools and processes that identify misconfigurations, policy drift, and exposure in cloud environments. It is strongest at discovery and weakest at enforcement, so it should be treated as a detection layer that feeds remediation rather than a control plane that changes access by itself.
• Execution Role: The IAM role an AI agent assumes to call cloud services on behalf of a workflow. It is the main control point for what the agent can do, so poor scoping turns an ordinary agent into a high-blast-radius identity with access that can outlive the original use case.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.