TL;DR: Enterprise AI use rose from 55% to 78% in a year, while AI-related security incidents hit a record 233 in 2024 and the average breach cost reached $4.88 million, according to Mindgard. The gap is now governance failure, not experimentation failure: security must move to discovery, red teaming, and lifecycle controls before AI becomes ambient infrastructure.
At a glance
What this is: This is a statistics-led analysis of AI security risk, showing that adoption, attack activity, and breach costs are all rising faster than defensive maturity.
Why it matters: It matters because IAM, security, and governance teams now need to treat AI systems, agents, and workflows as operational risk surfaces rather than side projects.
By the numbers:
- Enterprise AI use rose from 55% of organisations in 2023 to 78% in 2024, according to the Stanford HAI AI Index Report 2025 cited by Mindgard.
- AI-related security incidents reached a record 233 in 2024, according to the Stanford HAI AI Index Report 2025 cited by Mindgard.
- The average cost of an AI-driven breach reached $4.88 million, according to Harvard Business Review research cited by Mindgard.
Context
AI security now sits at the intersection of governance, application risk, and identity control. As organisations embed models and agents into business workflows, the question is no longer whether AI can be adopted, but whether its access, data use, and decision pathways are being governed with the same discipline applied to other high-risk systems.
The source article argues that the core problem is a readiness gap: organisations are scaling AI faster than they can secure it, and attackers are exploiting that gap with phishing, prompt injection, fake accounts, and model abuse. That makes AI security a programme issue, not a model-tuning issue.
Key questions
Q: How should organisations govern AI systems that can make consequential decisions?
A: Organisations should govern consequential AI systems with the same discipline used for high-risk identities: defined ownership, least privilege, logging, approval boundaries, and human override. The critical requirement is to connect model behaviour to real access paths so legal review, security review, and audit evidence all describe the same system.
Q: Why do AI-assisted attacks increase application risk so quickly?
A: AI-assisted tooling reduces the effort required to find exposed endpoints, vary payloads, and test credentials at scale. That means the same weakness can be exploited faster and more repeatedly, even if the underlying vulnerability is familiar. Teams should assume that attack volume and speed will rise before their detection and response processes are fully adapted.
Q: What are the signs that an AI security programme is failing against autonomous threats?
A: A programme is failing when assurance depends on quarterly assessments, when new tools or prompts appear without retesting, and when findings are only measured as patch backlog. Those signals show the team is still running on human-speed assumptions while the threat is moving continuously.
Q: What should teams do when an AI system is connected to business workflows?
A: Treat the connection as a governance decision, not a feature toggle. Define what the AI can access, what actions it may trigger, which approvals remain mandatory, and how you will detect misuse. If the workflow touches sensitive data or customer-facing processes, require risk review before expansion.
Technical breakdown
Why AI systems expand the attack surface differently
AI systems do not behave like conventional applications. They can accept natural-language input, retrieve data, call tools, and make decisions that depend on prompt context, model output, and surrounding integrations. That creates new failure modes, including prompt injection, sensitive-data leakage through prompts or files, and unsafe delegation to downstream tools. Security teams cannot rely on perimeter controls alone because the risk often appears inside the workflow, after the system has already accepted input or issued an action. See the AI Governance Guide.
Practical implication: Model, tool, and data access must be controlled as a single operating surface, not as separate security problems.
How agentic AI changes control assumptions
Agentic systems can sequence actions, call APIs, and pursue goals with less direct human supervision than a standard chatbot. That changes the trust model. Traditional approval chains assume a human can inspect a request before action occurs, but an agent can assemble context, invoke a tool, and pass results onward in one runtime loop. Once that loop is connected to production systems, the important question becomes who granted the agent authority, what it can reach, and how its actions are constrained. Read Red Teaming AI Agents for Identity Abuse.
Practical implication: Treat agent permissions, delegation boundaries, and tool scopes as the primary governance object, not the model prompt alone.
Why AI red teaming has become a control, not a test
AI red teaming is moving from validation exercise to operating control because organisations are discovering AI-specific issues only when adversarial testing is applied. The threats include prompt injection, data exfiltration through the model interface, hidden tool abuse, and harmful outputs that bypass conventional testing. Red teaming is useful because it probes behaviour under adversarial pressure rather than assuming the model will fail only in obvious ways. It also exposes where governance assumptions break when an AI system is integrated with identity, data, and workflow layers. See the AI Red Teaming Guide.
Practical implication: Use adversarial testing to verify how the AI system behaves under real abuse patterns before broader rollout.
Threat narrative
Attacker objective: The attacker aims to use AI systems as an acceleration layer for fraud, data exposure, or operational compromise.
- Entry occurs through AI-enabled phishing, prompt injection, or fake account creation that uses generative automation to bypass normal trust checks.
- Escalation follows when the attacker uses the model or agent as a workflow participant, pushing it to expose data, execute unsafe actions, or expand access beyond its intended scope.
- Impact is realised through credential theft, data exposure, fraudulent activity, or business disruption at machine speed.
Breaches seen in the wild
- DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.
- Microsoft SAS token exposure 2023: An over-permissive Azure SAS token in a Microsoft AI GitHub repo exposed 38TB, including workstation backups and Teams messages, for 3 years.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI governance debt is now an operational security problem: organisations are accumulating AI capability faster than they are building the controls needed to govern it. That means risk decisions are being deferred into production, where access, data handling, and delegation are already live. The consequence is that the first real governance test often arrives as an incident, not as a design review.
Shadow AI is the clearest named concept in this market phase: undiscovered models and agents create a governance blind spot that traditional inventories do not capture. If a team does not know where AI is embedded, it cannot reliably assess who can reach it, what data it sees, or which workflows it can influence. AI security maturity needs to include discovery and ownership, not just policy statements.
AI security is converging with identity governance: as models become workflow participants, access control shifts from user sessions to machine actions, API calls, and delegated authority. That makes identity and privilege boundaries central to AI risk, not adjacent to it. Practitioners should treat every AI integration as a governed identity relationship, not a standalone feature.
Machine-speed abuse changes the economics of defence: attackers using AI can scale phishing, scraping, account creation, and reconnaissance faster than human-led operations. That compresses detection windows and makes slow approval processes less effective against abusive automation. Security leaders should assume the defender now competes on telemetry, constraint, and response speed rather than on manual review alone.
Benchmark growth does not equal security maturity: the article shows model capability rising while readiness remains poor, and that mismatch is what creates board-level exposure. Stronger models can improve productivity and simultaneously widen attack paths if governance lags. Practitioners should measure AI security by controlled behaviour under attack, not by adoption or benchmark performance alone.
From our research library:
- Organisations that describe themselves as confident in their AI deployment actually experience a 72% security incident rate, compared to 33% for those who remain cautious, according to the 2026 Infrastructure Identity Survey.
- Only 13% of organisations feel extremely prepared for the reality of agentic AI despite the majority racing toward autonomous adoption, according to the 2026 Infrastructure Identity Survey.
- Read next: AI Security Maturity Model
What this signals
AI governance debt: when AI adoption runs ahead of control design, the programme inherits hidden exposure that only becomes visible under stress. Practitioners should expect the next wave of AI risk to come from mis-scoped access, unmanaged integrations, and the slow discovery of shadow AI across the environment.
The practical next step is to shift from project-by-project approval to lifecycle governance. That means inventory, ownership, testing, and change control must all move together, because a model or agent that is safe at pilot stage can become unsafe once it is connected to data, tools, and business decisions.
For practitioners
- Map every AI system and agent to an owner Build an inventory of models, agents, copilots, and embedded AI features across business units, then assign accountable owners for data access, tool use, and lifecycle decisions.
- Test AI workflows with adversarial scenarios Run red-team scenarios for prompt injection, data leakage, fake account abuse, and unsafe tool invocation before production rollout and after major workflow changes.
- Constrain AI delegation and tool scope Limit what each model or agent can call, read, write, or trigger, and review those boundaries as part of change control when the workflow expands.
- Tie AI governance to risk acceptance Document when an AI use case is approved, what data it can handle, and which compensating controls are required before it is allowed to affect customers or operations.
Key takeaways
- The central risk is not AI adoption itself, but the gap between rapid deployment and the controls needed to govern access, data use, and delegated actions.
- The article’s evidence shows a steep rise in AI use alongside a record level of incidents and multi-million-dollar breach costs, which makes the issue operational rather than theoretical.
- Security teams need inventory, ownership, adversarial testing, and scoped delegation if they want AI systems to remain governable as they scale.
Related NHI Mgmt Group resources: AI Security Maturity Model · AI Governance Guide · AI Risk Assessment Guide · AI Red Teaming Guide
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article is fundamentally about governance lagging behind AI adoption and risk. |
| Recommendation — Define AI ownership, approval, and accountability under GOVERN before production rollout. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Agentic systems and delegated workflows are central to the article's risk discussion. |
| ASI02 — Tool Misuse | The article repeatedly describes unsafe tool calls and workflow abuse. | |
| Recommendation — Restrict agent privileges and delegated actions to prevent identity and privilege abuse. Constrain tool access and validate every high-risk action an agent can initiate. | ||
| MITRE ATLAS | TA0006 — Credential Access | The article highlights AI-enabled phishing, fake accounts, and data theft patterns. |
| Recommendation — Map AI-enabled credential theft and phishing scenarios to TA0006 in your threat model. | ||
| NIST CSF 2.0 | GV.RM-01 — Risk Management Strategy | The piece frames AI risk as a programme-level governance and risk-management issue. |
| Recommendation — Embed AI risk into enterprise risk strategy and review it at the same cadence as other top risks. | ||
Key terms
- Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
- Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
- AI governance debt: The accumulation of AI risk when deployment decisions outpace the controls needed to manage them. It appears when organisations scale models and agents before they have ownership, inventory, testing, approval, and lifecycle rules in place, leaving security teams to govern after exposure has already spread.
- AI Red Teaming: AI red teaming is the practice of simulating hostile behaviour against models, applications, and agents to expose weaknesses before real attackers do. In AI programmes, it is most useful when results can be turned into controls, monitoring, and governance evidence rather than left as a one-time test report.
Deepen your knowledge
The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity controls to the broader AI and security programmes they already run.
Published by the NHIMG editorial team on October 22, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org