AI data security now fails at prompts, agents and APIs

At a glance

What this is: Mindgard argues that AI is turning data security into a visibility and control problem across prompts, agents, APIs, model pipelines and supply chains.

Why it matters: For IAM and security teams, the key issue is that AI systems now move data through identities and integrations faster than legacy controls can govern, especially where access, privilege and third-party exposure overlap.

By the numbers:

• 68% of organisations have already faced data leaks related to employee use of AI tools, while only 23% have AI security policies that govern employee use.
• 39.7% of total interactions with AI tools include prompt text, copy-paste functions and file uploads that contain sensitive data.
• In 2025, the FBI IC3 received 22,364 AI-related complaints from victims that cost Americans almost $893 million.

Context

AI data security now extends well beyond traditional DLP and perimeter controls. Prompts, agents, APIs and model pipelines move sensitive data across systems in ways that are faster, less visible and harder to govern than older enterprise workflows.

That shift matters because the security boundary is no longer just the application or endpoint. It now includes the identity, permissions and data paths of AI systems themselves, which makes AI governance inseparable from agent identity and privilege abuse when systems can request, route or expose data autonomously.

Key questions

Q: What breaks when AI data access is not centrally governed?

A: When access is fragmented, teams lose visibility into what data is feeding models, who can change it, and whether outputs can be trusted. That leads to leakage, weak accountability, and compliance evidence that cannot be defended. Central governance is the only way to connect data controls to AI outcomes.

Q: Why do AI agents make excessive access more dangerous than human access?

A: AI agents can use inherited permissions continuously, across multiple systems, and at machine speed. That means a single overbroad entitlement can create rapid exposure to data, workflows, or administrative actions before humans notice. The risk is driven by both breadth of access and the pace at which the agent can exercise it.

Q: What are the signs that traditional controls are failing to see shadow AI use in the enterprise?

A: A common sign is that AI activity appears in business outcomes but not in security logs. If users can access AI tools through personal accounts, paste text into browser tabs, or run AI assistants inside development environments without clear audit trails, perimeter monitoring and file-based DLP are not enough. Those gaps usually indicate the organisation is observing traffic, not behaviour.

Q: How do organisations balance AI data use with privacy and compliance requirements?

A: Organisations should tie AI data use to explicit governance. That means defining which datasets are allowed for AI, applying native data controls, reviewing third party risk where AI services are involved, and reassessing exposure as models or workflows change. The practical objective is to support innovation without expanding access to personal or regulated data beyond what is necessary.

Technical breakdown

Why prompts and agents create a new data exposure layer

Prompts and agent interactions create a control plane for data movement that traditional DLP often cannot inspect well. Users can paste sensitive text into consumer and enterprise AI tools, upload files, or let agents transform data across multiple services in one workflow. The problem is not only content leakage. It is the combination of speed, scale, and context loss, where security teams may not know which identities, files, or downstream systems the AI touched. Once that visibility is missing, policy enforcement becomes reactive instead of preventive. See the Generative AI Data Loss Prevention Guide.

Practical implication: treat prompt and agent traffic as governed data flow, not just user activity.

Why AI systems need identity and access control overhaul

AI systems often inherit too much privilege from the humans and services around them. When an AI tool or agent can access files, APIs, and SaaS systems beyond the task at hand, the blast radius of compromise expands quickly. This is an identity problem as much as a data problem, because the AI system becomes a non-human actor with effective access rights. Continuous authentication, least privilege, and scoped delegation matter more than static policy language when the system itself can decide where to send information next. Read Red Teaming AI Agents for Identity Abuse.

Practical implication: review AI entitlements the same way you review privileged non-human access.

How AI supply chains and integrations become exfiltration paths

AI supply chains include models, plugins, tools, APIs, and third-party extensions that can move data outside the organisation’s direct control. Malicious or poorly governed integrations can masquerade as legitimate services while forwarding sensitive information to attacker-controlled destinations. This matters because modern AI stacks are deeply interdependent: a trusted connector, a compromised package, or an over-scoped token can turn a normal workflow into a covert data path. Security teams need to think in terms of provenance, trust boundaries, and runtime behaviour, not only vendor approval lists. See the AI Supply Chain Security and AI-BOM Guide.

Practical implication: inventory AI integrations and validate what each one can access, send, and retain.

Threat narrative

Attacker objective: The attacker wants to extract sensitive data, extend access through trusted AI workflows, and persist inside ordinary business processes without detection.

1. Entry occurs when users, agents, plugins, or third-party AI services receive sensitive data through prompts, uploads, or connected workflows.
2. Credential and privilege abuse follow when the AI system or attached integration can access more data, APIs, or services than the task requires.
3. Exfiltration happens when the workflow routes sensitive content to external systems, attacker-controlled endpoints, or shadow AI services outside security visibility.
4. Impact is data leakage, compliance exposure, and broader operational disruption when the organisation cannot reliably see what the AI system accessed or shared.

Breaches seen in the wild

• postmark-mcp malicious MCP server 2025: A fake Postmark MCP server on npm quietly BCC'd every email AI agents sent through it to an attacker, using victims' own Postmark tokens.
• xinference PyPI compromise 2026: Three xinference releases on PyPI carried a stealer for cloud, Kubernetes and registry keys, reportedly via a compromised bot account.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI data security has become an identity governance problem, not just a content filtering problem. Once an AI system can access files, APIs, or SaaS tools, its privilege profile becomes part of the organisation’s attack surface. The article is right to frame prompts, agents, and integrations as separate exposure layers, because each layer changes who or what can move data next. Practitioners should stop treating AI as a passive application and manage it as a governed actor with scoped authority.

Legacy DLP is structurally incomplete for AI workflows. DLP was built to inspect relatively stable data paths, not to understand an agent that can summarise, transform, retrieve, and re-route information across multiple services in one session. That means the control gap is not only visibility but timing: by the time policy enforcement sees the leak, the data may already have moved. The right response is runtime governance, not a hope that static rules will keep up.

Shadow AI exposes a governance debt that most enterprises have not yet measured. The problem is not limited to banned consumer tools. It includes sanctioned platforms, browser extensions, agent frameworks, and third-party connectors that expand the data path without equivalent review. Organisations that do not map those dependencies will underestimate both their breach surface and their compliance burden. A practical AI programme needs inventory, approval, and revocation discipline before it can claim control.

AI red teaming is becoming a control validation layer for sensitive data protection. The article’s central lesson is that AI systems must be tested in the same adversarial way as other high-change, high-privilege assets. That is especially true where sensitive data, delegated access, and third-party integrations intersect. Security teams should treat continuous testing as a governance requirement, not a niche assessment exercise.

Named concept: prompt-to-exfiltration drift. This is the tendency for a harmless-looking user prompt to trigger data movement across agents, connectors, and APIs that security teams never intended to allow. It matters because the security decision is no longer just whether a user can type something. It is whether the surrounding AI workflow can preserve trust boundaries after the prompt is accepted. Practitioners should govern the entire data path, not the prompt alone.

From our research library:

• 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, according to the State of Secrets in AppSec.
• 92% of organisations expose NHIs to third parties, raising concerns about supply chain security, according to the Ultimate Guide to NHIs.
• Read next: Data Security Guide

What this signals

Prompt-to-exfiltration drift: AI governance now has to account for the moment a prompt becomes a data-moving workflow, because the risky event is often not the request itself but the chain of retrieval, transformation, and export that follows. Security teams should design controls around data paths, connector trust, and identity scope rather than around the prompt text alone.

The operational signal is clear: organisations need to shift from policy-only AI oversight to runtime enforcement, especially where agents, plugins, and third-party systems can move information outside approved boundaries. A control set built for static applications will not reliably contain dynamic AI behaviours, even when the workload looks familiar on the surface.

For practitioners

• Map AI data paths and trust boundaries Document every place sensitive information can enter, transform, or leave an AI workflow, including prompts, file uploads, agents, plugins, and API calls.
• Scope AI entitlements to the task Remove standing access that is not required for the current use case, and separate read, write, retrieval, and external sharing permissions.
• Block shadow AI and unsanctioned connectors Detect unmanaged AI tools, browser extensions, and third-party integrations, then revoke or isolate anything that bypasses approved identity, logging, or data controls.
• Validate AI exposure with adversarial testing Use red teaming to test prompt injection, data leakage, over-privileged agents, and unsafe connector behaviour before production changes expand the blast radius.

Key takeaways

• AI is expanding the data security problem into prompts, agents, APIs, model pipelines, and supply chains, which makes visibility and access governance the first control challenge.
• The article shows that adoption is already outrunning policy, with most organisations reporting AI-related leakage while only a minority have formal AI use policies in place.
• The practical response is to govern AI workflows as privileged, dynamic systems and to validate them continuously rather than relying on perimeter tools or annual reviews.

Related NHI Mgmt Group resources: Data Security Guide · Red Teaming AI Agents for Identity Abuse · AI Supply Chain Security and AI-BOM Guide · SaaS-to-SaaS and OAuth App Governance Guide

Key terms

• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Prompt-to-exfiltration drift: Prompt-to-exfiltration drift is the shift from an apparently harmless AI prompt to an unintended data transfer across tools, connectors, or external services. The risk is not the prompt alone, but the chain of automated retrieval, transformation, and forwarding that can move sensitive data beyond control boundaries.
• AI Data Privacy: AI data privacy is the discipline of protecting personal and regulated information as it moves through training, tuning, inference, and operational use. It focuses on preventing exposure, memorization, misuse, and unauthorized reuse of sensitive data in AI systems that process large, varied, and constantly changing inputs.
• Non-Human Identity (NHI): A digital identity assigned to a non-human entity such as a software application, service account, API key, bot, machine, or AI agent that enables it to authenticate and interact with systems without direct human involvement. NHIs now outnumber human identities in most enterprises by 25 to 50 times.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity and secrets management for teams building control around identity-driven systems. It helps security practitioners connect identity discipline to the broader governance work that complex AI and cloud programmes now require.