AI security trends show adoption outpacing governance controls

At a glance

What this is: This article surveys AI security trends and finds that enterprise adoption, threat activity, and regulation are all moving faster than security readiness.

Why it matters: For IAM, NHI, and AI security teams, the key issue is that governance must catch up with how AI is actually being used across business functions, not just with model rollout.

By the numbers:

• 49% of firms use tools like ChatGPT across departments, from IT and marketing to finance and customer service.
• 77% of organisations find themselves unprepared to defend against AI threats.
• 90% of organisations are actively implementing or planning to explore LLM use cases.
• Only 5% of security professionals say their organisation is highly confident in its AI security preparedness.

Context

AI security now sits at the intersection of adoption, governance, and operational risk. The article argues that businesses are using generative AI widely, but their controls are not maturing at the same pace, which creates a widening gap between what users can do and what security teams can reliably govern.

For identity and access programmes, the relevant question is not whether AI is being adopted, but whether access, policy, and data controls can keep pace with how AI tools are embedded into day-to-day work. That matters for human users, NHI-backed integrations, and emerging AI agent workflows alike.

The article’s figures point to a familiar pattern in identity security: usage expands through ordinary business demand, while governance remains fragmented across policy, privacy, and security teams. That makes AI security a programme issue rather than a point-control issue.

Key questions

Q: How should security teams enforce AI acceptable use policies at runtime?

A: Security teams should pair the written policy with discovery, intent-based controls, and audit logging. The policy defines what is allowed, but runtime enforcement decides whether a prompt is warned, blocked, routed, or recorded. Without that layer, employees can bypass the document through normal work patterns, and the organisation cannot prove what happened during an AI interaction.

Q: Why do AI tools create governance risk even when humans stay in charge?

A: AI tools create risk when they reshape the real decision path without changing formal ownership. Teams may rely on output that is faster, more persuasive, or less scrutinised than human work. The result is weaker accountability, not because AI is autonomous, but because the control process stops matching how decisions are actually made.

Q: What are the signs that AI is not being governed well in a security program?

A: Warning signs include unclear ownership, no formal review of AI use cases, weak vendor due diligence, and limited ability to explain how decisions are made or tested. If leadership cannot document controls, assess risk tiers, or describe how failures and breaches will be disclosed, AI governance is immature. Those gaps usually show up before a security incident.

Q: What should organisations do when AI regulation and adoption move at different speeds?

A: They should build internal controls that can absorb regulatory change without rebuilding the entire programme each time. That means clear accountability, documented use cases, and policy enforcement that can adapt as legal requirements evolve across privacy, security, and sector-specific obligations.

Technical breakdown

Why AI security controls lag behind adoption

AI adoption often spreads through shadow use, departmental experimentation, and embedded workflow tools before security teams can define policy boundaries. That creates a control problem across authentication, data handling, and permitted use, because the same model may be accessed by different users, applications, and business units with inconsistent oversight. In identity terms, the issue is not just who signs in, but who can prompt, connect, export, and operationalise outputs. Governance breaks when access policy is treated as a one-time approval instead of a living control surface.

Practical implication: security teams need to treat AI access as a governed capability with defined usage boundaries, not just a software entitlement.

What makes AI threats different from ordinary application risk

AI systems create a blended attack surface that includes prompt injection, jailbreaks, data leakage, harmful output generation, and downstream misuse of model-connected tools. That means defenders have to think about both the model layer and the identity layer, because attackers often aim to manipulate behaviour rather than exploit traditional code flaws. The article’s concern about dynamic content and AI-powered attacks reflects this shift. For practitioners, model access, tool access, and data access cannot be governed as separate problems if the system acts on behalf of a user or workflow.

Practical implication: map AI attack paths across identity, data, and tool integration before deciding which controls belong in policy, runtime, or monitoring.

How regulation is reshaping AI governance expectations

The article notes that regulation is expanding alongside adoption, especially in the EU and the US, which pushes AI from an innovation discussion into a governance discipline. That matters because regulatory pressure tends to force clearer accountability for data handling, model oversight, and security obligations. Security teams should expect AI governance to converge with privacy, risk, and identity policy instead of living in a separate innovation track. The organisations that wait for a complete standards landscape usually end up reacting to local policy exceptions and fragmented controls.

Practical implication: align AI policy, privacy review, and security governance now so controls can be adapted as regulatory obligations harden.

Threat narrative

Attacker objective: The objective is to exploit weak AI governance to reach sensitive data, influence outputs, or increase the success rate of broader phishing and social engineering campaigns.

1. Entry occurs through widespread, ordinary use of GenAI tools across departments, including unsanctioned or weakly governed access paths into business workflows.
2. Credential or policy abuse follows when users, integrations, or connected tools can move data into AI systems without tight control over what is submitted or returned.
3. Impact appears as data exposure, insecure outputs, and an expanded attack surface that security teams cannot defend consistently with conventional controls.

Breaches seen in the wild

• 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI security governance is now an access problem, not just a model-risk problem: once tools like ChatGPT are used across departments, the real control question becomes who can use them, what data they can reach, and what actions outputs can trigger. That crosses IAM, data governance, and application policy boundaries. Treating AI as a standalone innovation silo leaves the access model incomplete, so practitioners need governance that follows usage rather than vendor rollout.

Identity controls built for static applications do not fit dynamic AI behaviour: conventional approvals assume a stable user, stable purpose, and stable data boundary. AI usage breaks those assumptions because the same interface can be reused across teams, prompts, integrations, and downstream automations. The named concept here is runtime governance gap: policy exists on paper, but enforcement lags the moment when model interaction actually happens.

The security challenge is widening because AI amplifies both human and machine misuse paths: employees can expose sensitive information through prompts, while connected services can turn the same model into a propagation channel for bad decisions or data movement. That means identity governance must cover human use, NHI-linked integrations, and emerging autonomous behaviour in one operating model. Practitioners should stop treating AI controls as a narrow content-filtering exercise.

Regulation will not close the gap by itself: the article’s regulatory trendline shows why formal requirements are arriving faster than operational maturity. Compliance pressure may force clearer accountability, but it will not solve fragmented ownership between security, privacy, and business teams. The organisations that will cope best are those that can translate policy intent into access boundaries, monitoring, and approval workflows that work in production.

AI security is becoming a governance maturity test for the entire identity programme: the organisations that can define acceptable use, map data boundaries, and monitor AI-connected actions will separate risk containment from unchecked adoption. That applies equally to human users and the non-human services that carry prompts, context, and outputs between systems. The practical conclusion is straightforward: AI security now measures whether identity governance can operate at business speed.

From our research library:

• Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.

What this signals

Runtime governance gap: AI security fails fastest when policy assumes a stable application lifecycle, but users can adopt, share, and operationalise AI tools before review cycles catch up. The programme response is to govern usage at the point of interaction, not after broad adoption has already occurred.

AI security is converging with IAM, privacy, and data governance because the main risk is no longer just model misuse, but uncontrolled access to data and business processes through AI-connected workflows. Teams that separate those domains will keep missing the operational boundary where risk actually appears.

For practitioners

• Define AI usage boundaries by data class Classify which data types may enter public, enterprise, and embedded AI tools, then enforce the policy through DLP, application controls, and user guidance.
• Inventory AI-connected access paths Map every internal app, browser extension, workflow, or service account that can submit prompts or consume model output, including sanctioned and unsanctioned paths.
• Separate human and non-human AI use cases Document where people interact directly with AI and where services, automations, or integrations do so on their behalf, because the control model differs.
• Add monitoring for prompt and output misuse Review logs for sensitive prompt content, repeated policy exceptions, unexpected data exports, and model responses that trigger downstream actions.

Key takeaways

• AI adoption is now broad enough that security teams can no longer treat it as a niche innovation concern.
• The sharpest risk is the gap between who can use AI and what those users and integrations can do with data and outputs.
• Governance has to move from policy statements to enforceable access boundaries, monitoring, and accountability across human and non-human use cases.

Key terms

• API Security Governance: API security governance is the set of policies, controls, and oversight used to keep application programming interfaces safe and accountable. It defines who can create, expose, use, monitor, and retire APIs, then enforces authentication, authorization, logging, rate limits, schema validation, and lifecycle review across internal and external integrations.
• Governance Gap: A governance gap is the distance between knowing an asset exists and being able to enforce policy on it. In identity programmes, it appears when discovery, review, and enforcement are split across different tools or teams, leaving access partially visible but not truly controlled.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• AI-Connected Workflow: An AI-connected workflow is any process in which an AI system, model, or agent can read, transform, or route sensitive data as part of business operations. The governance challenge is that these workflows often inherit access from humans or services without a clear business justification for every permission.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.