GenAI security statistics show governance is lagging adoption

At a glance

What this is: This is a statistics-led analysis of GenAI security risk showing that sensitive data exposure, prompt injection, and weak governance coverage are now mainstream issues.

Why it matters: It matters because IAM, IGA, and security teams need control models that cover AI-driven data access, policy enforcement, and response readiness, not just human authentication and traditional DLP.

By the numbers:

• 4% of prompts and over 20% of files uploaded to GenAI tools contain sensitive corporate data.
• Prompt injection remains a significant threat, with 56% of tests across 36 LLM configurations successfully bypassing safeguards.
• 75% of companies have AI policies, but fewer than 60% have invested in trained governance staff or incident-response capabilities.

👉 Read Knostic’s full statistics roundup on GenAI security risk and governance gaps

Context

GenAI security is increasingly an identity and governance problem, not just a model safety problem. Once employees can paste sensitive material into chat interfaces or connect AI tools to enterprise data, the control question becomes who can access what, under which policy, and with what monitoring.

The article’s core message is that adoption has outpaced enforcement. Traditional DLP, policy documents, and awareness training do not close the gap when prompts, uploads, and model output become part of the data path that security teams must govern.

Key questions

Q: How should organisations control sensitive data in GenAI tools?

A: Organisations should treat prompts, uploads, and model outputs as governed data flows, then apply classification, inspection, and logging at the point of use. The control objective is to stop sensitive information from entering AI workflows without visibility. That requires policy, access rules, and monitoring to work together, not as separate programmes.

Q: Why do prompt injection attacks remain hard to stop?

A: Prompt injection is hard to stop because the model may not reliably distinguish trusted instructions from malicious content once both are present in the same context. Defenses must therefore be layered, including input filtering, context isolation, tool restrictions, and adversarial testing. A prompt policy alone does not provide runtime assurance.

Q: What do security teams get wrong about AI governance policies?

A: The common mistake is assuming that a written policy equals enforcement. Real governance needs assigned owners, escalation paths, evidence collection, and response playbooks that work during misuse or leakage. Without those operational pieces, policy becomes a compliance artifact rather than a control.

Q: How can teams measure whether GenAI controls are actually working?

A: Teams should measure exposure rates, blocked transactions, test failures, and incident response readiness, then compare those signals over time. If sensitive data still appears in prompts or uploads, or if adversarial tests bypass safeguards, the control design is not yet effective. The metric must reflect runtime behaviour, not policy presence.

Technical breakdown

Sensitive data exposure in prompts and uploads

GenAI tools create new data-handling pathways where employees can move regulated, proprietary, or confidential content into systems that were not originally designed as enterprise repositories. Prompt text and file uploads become an access channel, which means the exposure problem sits at the intersection of identity, classification, and policy enforcement. If the control plane does not inspect or classify content at the point of use, security teams lose visibility into what entered the AI workflow and why. That makes incident investigation, legal review, and containment materially harder.

Practical implication: enforce content inspection and access controls at the prompt and upload boundary, not only at storage and email layers.

Prompt injection and policy bypass

Prompt injection works by manipulating model behavior through crafted instructions that override or confuse intended safeguards. In practice, this turns the model into an execution path for attacker-controlled text, especially when the AI can retrieve data, call tools, or summarize privileged content. The 56% bypass rate across tested configurations shows that defensive prompts alone are not a durable control. Effective protection depends on layered filtering, context isolation, and monitoring of tool-using AI workflows where the model can be steered into disallowed actions.

Practical implication: treat prompt injection as a control-failure class and validate protections with adversarial testing, not policy statements.

AI governance staffing and incident response gaps

A policy without trained owners and playbooks is documentation, not governance. The article shows that many organisations have written AI policies while fewer have the staff, escalation paths, and response procedures needed to enforce them during real incidents. That matters because AI risk spans legal, compliance, security, and business teams, and each needs a defined role when misuse, leakage, or adversarial content appears. Without ownership, exceptions linger and evidence disappears.

Practical implication: assign accountable AI governance roles and rehearse incident response for data leakage, unsafe outputs, and policy violations.

NHI Mgmt Group analysis

AI governance is becoming an identity governance problem because data access now happens through conversational systems. When employees can move sensitive data into GenAI tools, the security question is no longer limited to login and entitlement management. It becomes a question of who is allowed to reveal, retrieve, or transform information through an AI interface, and how that behaviour is measured and contained. Practitioners should treat GenAI access paths as governed identity surfaces, not as informal productivity tools.

Prompt injection exposes a governance gap that policy documents cannot close on their own. The article’s 56% bypass figure shows that model-facing controls remain porous across many configurations. That does not mean every deployment is equally exposed, but it does mean that control assurance has to be tested as a runtime property, not assumed from the presence of written rules. Practitioners should evaluate whether their AI controls are actually resistant to adversarial instruction, not merely documented.

Oversharing risk is now a data-boundary problem, not just a user-behaviour problem. The article’s finding that more than 20% of uploaded files contain sensitive corporate data points to a structural issue in how organisations extend trust into AI workflows. When the AI tool becomes a destination for documents and prompts, existing classification and DLP controls must follow the data into that path. Practitioners should map which content types are permitted in AI interactions and which are not.

Policy adoption without trained governance staff creates an enforcement illusion. The article shows that many organisations can point to AI policies but cannot yet operationalise them through owners, escalation paths, and incident handling. That gap is especially important for identity teams because AI governance spans entitlement decisions, logging, compliance review, and response coordination. Practitioners should not measure readiness by policy presence alone.

GenAI security statistics should now be used as an operating metric for programme design. The useful signal here is not just that AI creates new risk, but that the risks are quantifiable enough to drive policy thresholds, control selection, and executive reporting. That is where identity, data security, and response planning converge. Practitioners should use measured exposure and bypass rates to prioritise controls that reduce real attack surface, not theoretical concern.

From our research:

• 33% of organisations report their AI agents have accessed inappropriate or sensitive data beyond their intended scope, according to the AI Agents: The New Attack Surface report.
• 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• That same research shows 80% of organisations report AI agents have already performed actions beyond their intended scope, which is why OWASP NHI Top 10 remains a useful forward reference.

What this signals

Oversharing in GenAI is now measurable enough to drive access policy, not just awareness messaging. When more than 20% of uploaded files carry sensitive corporate data, the control problem shifts from user education to workflow design. Identity teams should work with data security owners to decide which content types are permitted in AI interactions and which require explicit blocking or redaction.

AI governance maturity will increasingly be judged by enforcement artefacts rather than policy counts. The organisations that can prove monitoring, escalation, and exception handling will be better positioned than those that only publish acceptable-use rules. That is why security leaders should align their programmes to external baselines such as the OWASP Agentic AI Top 10 and test controls against live workflows.

Prompt injection is a named concept practitioners should now track as an operational boundary failure. It describes the moment when hostile text steers a GenAI system into ignoring intended restrictions or exposing data. Teams that run retrieval-augmented or tool-using AI should treat that failure mode as part of their control testing and incident planning.

For practitioners

• Classify GenAI prompt and upload paths as governed data flows Map which prompts, files, and outputs may contain sensitive corporate data, then apply approval, inspection, and logging controls at the point of interaction. Align the workflow to your data classification rules so AI use does not become an uncontrolled side channel.
• Test prompt-injection resilience with adversarial scenarios Run structured tests against the models and AI workflows you actually use, including retrieval, summarisation, and tool invocation. Measure whether safeguards fail under malicious instructions rather than assuming prompt-based guardrails are sufficient.
• Assign accountable AI governance owners Name the security, legal, compliance, and operations roles that own policy enforcement, exception handling, and incident response for GenAI use. Tie those roles to escalation paths and review cycles so the policy has an operational owner.
• Build evidence-ready reporting for AI risk Track sensitive data exposure rates, blocked transactions, and incident outcomes in a form that supports board reporting and audit review. Use those measures to decide where access controls, monitoring, or staff training need to be tightened first.

Key takeaways

• GenAI risk is no longer abstract because sensitive corporate data is already entering prompts and file uploads at meaningful rates.
• Prompt injection remains a practical bypass path, so policy-only controls do not provide enough assurance for production AI workflows.
• The deciding factor for AI readiness is enforcement, including ownership, monitoring, and response, not the existence of a written policy.

Key terms

• Prompt Injection: Prompt injection is a technique that manipulates a GenAI system by placing malicious instructions into text the model treats as input or context. The result can be policy bypass, unsafe output, or unintended data exposure when the system cannot reliably separate user intent from attacker-controlled content.
• AI Governance: AI governance is the set of policies, roles, controls, and evidence practices used to manage how AI is deployed and operated. In security terms, it turns AI use from ad hoc experimentation into an accountable process with ownership, monitoring, escalation, and auditability.
• AI-specific Data Loss Prevention: AI-specific data loss prevention is the application of content controls to prompts, uploads, outputs, and connected AI tools. It goes beyond traditional DLP by focusing on conversational and model-mediated data movement, where sensitive information can be exposed before it ever reaches a storage system.

What's in the full report

Knostic's full article covers the source studies and implementation details this post intentionally leaves for the original:

• The underlying research methodology behind each statistic, including how the sources defined prompts, uploads, and incidents
• The specific control recommendations Knostic ties to AI-specific DLP, prompt sanitization, and governance staffing
• The report’s discussion of AI policy enforcement gaps and how teams can operationalise response readiness
• The vendor’s examples of AI security use cases across Copilot, search, and other enterprise GenAI workflows

👉 Knostic’s full article includes the source studies, definitions, and implementation context behind each statistic

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.