Shadow AI governance gaps expose sensitive data at scale in Australia

At a glance

What this is: Josys' report shows that shadow AI is already driving sensitive data exposure in Australian organisations, with workers uploading company information into AI tools faster than governance can keep up.

Why it matters: IAM, IGA, and security teams need visibility, policy enforcement, and audit-ready controls because unauthorised AI use turns ordinary productivity behaviour into unmanaged data leakage and compliance risk.

By the numbers:

• Over a third (36%) of employees upload sensitive company information to AI tools.
• 70% of organisations have little to no visibility into what AI tools are actually being used.
• 63% of users lack confidence in using AI securely.
• One in four users (24%) admit to uploading customer PII.

Context

Shadow AI refers to employees using unauthorised AI platforms outside approved governance, policy, and oversight. In this report, the governance gap is not limited to model risk or security tooling, but to the absence of visibility into where sensitive company data is being sent, by whom, and under what controls.

The Australian survey shows that AI adoption is ahead of organisational readiness. That matters for IAM and data governance programmes because access decisions, policy enforcement, and audit trails are no longer confined to sanctioned systems. When workers can move sensitive information into external AI platforms on their own, the identity perimeter expands beyond the tools the organisation actually controls.

Key questions

Q: What breaks when employees use shadow AI for work tasks?

A: Shadow AI breaks identity visibility and lifecycle control. Employees can create or use AI accounts, connect them to work data, and move sensitive information outside approved governance. That leaves security teams unable to reliably inventory the identity, review its access, or revoke delegated permissions when the business need ends.

Q: Why does shadow AI create compliance and accountability gaps for organisations?

A: Shadow AI breaks the link between business activity, policy approval, and audit evidence. When employees use unapproved models or chatbots, sensitive data can leave controlled environments and outputs may influence decisions without traceability. That makes it harder to prove compliance, assign accountability, reconstruct events during an investigation, and show that data handling met regulatory expectations across the organisation.

Q: How can organisations tell whether AI governance is actually working?

A: Organisations can tell AI governance is working when they can inventory every agent, explain its purpose, show who owns it, and prove that permissions are tightly scoped. If those four things are missing, the programme has policy language but not operational control. Auditors will notice the gap quickly.

Q: What should security teams do when AI tools appear outside approved channels?

A: Treat the finding as a governance exception, not just a procurement issue. Determine which users, data types, and business units are affected, then bring discovery, policy, and logging into a single control process before the behaviour becomes routine.

Technical breakdown

Shadow AI creates an identity and data governance perimeter outside approved SaaS

Shadow AI is unauthorised AI platform use that sits outside the normal SaaS approval and access governance process. The security problem is not only the application itself, but that employees can move sensitive data into a third-party service without it appearing in sanctioned inventory, recertification, or policy enforcement workflows. That leaves IAM and IGA teams managing identities inside the organisation while the most sensitive action happens in an ungoverned destination. Practical implication: treat unsanctioned AI platforms as part of your access surface, not just an application-choice issue.

Practical implication: extend access governance to unsanctioned AI usage and not just sanctioned applications.

Why visibility failures turn AI use into audit and compliance exposure

Visibility is the control that tells you which tools are being used, who is using them, and what data types are flowing through them. Without that, policy enforcement becomes symbolic because you cannot prove whether employees are handling customer PII, financials, or internal strategy in approved ways. In regulated environments, lack of visibility also weakens incident scoping, because teams cannot reconstruct which data left the approved environment. Practical implication: build discovery and logging around AI tool usage before attempting to enforce data handling policy.

Practical implication: instrument discovery and logging before relying on policy enforcement.

Role-based policy only works when data sensitivity and user function are tied together

The report points to role-based access and risk tiers as the basis for real-time policy. That is important because not all AI use is equally risky: a finance user handling financials, a healthcare user handling PII, or a sales user handling customer data creates different governance obligations. Attribute-based policy and data classification therefore matter as much as account provisioning. The control failure is not merely lack of rules, but lack of policy context at the moment data is shared. Practical implication: align policy decisions to role, data type, and business function in one governance layer.

Practical implication: use role, data sensitivity, and business function together in policy decisions.

Threat narrative

Attacker objective: The objective is usually not a direct compromise but uncontrolled access to valuable company information through unmanaged AI platforms and the resulting governance failure.

1. Entry occurs when employees adopt unauthorised AI platforms and move work data into them outside approved governance processes.
2. Sensitive information is then uploaded into the AI service, including strategy documents, financials, customer PII, and legal or compliance material.
3. The organisation loses control over where the data resides, who can access it, and whether retention or reuse settings match policy.
4. Impact follows as compliance exposure, audit blind spots, and potential data leakage across tools the organisation never sanctioned.

Breaches seen in the wild

• Vercel Context.ai OAuth Supply Chain Breach: Shadow AI app Context.ai OAuth integration exposes Vercel customer data via unmanaged third-party token.
• Salesloft OAuth token breach: hackers stole OAuth tokens to access Salesforce data via Salesloft.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Shadow AI is no longer just an acceptable-use problem. It is an identity governance problem because the user is still authenticated, but the destination platform sits outside the organisation's approved control plane. Once that happens, access reviews, policy enforcement, and audit evidence no longer describe the full data path. Practitioners should treat unmanaged AI use as a governed access domain, not a fringe productivity behaviour.

Shadow AI creates governance debt at the point of disclosure. The organisation may still control endpoint access and SaaS provisioning, yet it cannot govern what happens after a user copies regulated or confidential data into an external model. That means the control failure is not only visibility but loss of enforceable context at the moment of data sharing. The right response is to define the disclosure boundary as part of IAM and IGA scope, not as a separate AI project.

Data classification must now drive access policy beyond the enterprise perimeter. A strategy document, customer PII, and legal material each carry different exposure consequences once they leave approved systems. If role-based policy stops at internal apps, it will miss the most consequential transfer point in shadow AI use. Security teams should reframe policy as an identity-plus-data decision, because the risk is created when a trusted user can move sensitive content into an untrusted platform.

Unified AI governance is the named concept this report exposes. Visibility, policy enforcement, and automation only work when they are treated as one operating model rather than three separate controls. In practice, fragmented ownership between IAM, security, privacy, and business teams is exactly what lets shadow AI persist. Practitioners should align governance, discovery, and enforcement under a single control model that covers user identity and downstream data handling.

Smaller organisations face a different version of the same control gap. The survey shows they are less equipped to assess AI risk, which means informal adoption can outpace even basic oversight faster than in larger enterprises. That makes manual review and ad hoc approval especially weak as operating models. Security leaders should assume shadow AI will appear first where governance maturity is lowest, then spread laterally across the business.

From our research library:

• 92% of organisations expose NHIs to third parties, raising concerns about supply chain security, according to the Ultimate Guide to NHIs.
• Read next: Shadow AI and AI Agent Discovery Guide

What this signals

Unified AI governance is the practical lesson here: organisations need a single control model that covers discovery, policy enforcement, and data-handling oversight before employees normalise shadow AI use. Without that, IAM and privacy teams will keep reacting after the data has already left sanctioned systems.

The survey's 70% visibility gap shows why discovery is the first governance threshold, not the last. Once an organisation cannot see which AI tools are in use, every later control depends on guesswork rather than policy evidence.

For practitioners

• Establish AI usage discovery Identify which AI platforms employees are actually using across browsers, devices, and managed SaaS estates, then reconcile that list against approved applications and procurement records.
• Classify data before disclosure Map sensitive data types such as customer PII, financials, strategy documents, and legal material to explicit handling rules before users can submit them to AI tools.
• Enforce role-based AI policy Tie policy decisions to role, business function, and sensitivity tier so that finance, healthcare, sales, and technical users do not share under the same rule set.
• Make shadow AI auditable Require logging, reviewable policy decisions, and exception handling for AI tool use so security and compliance teams can reconstruct data movement after the fact.
• Assign ownership for AI governance Define which teams own discovery, policy enforcement, privacy review, and user education so shadow AI does not become a cross-functional gap with no accountable controller.

Key takeaways

• Shadow AI turns everyday productivity behaviour into an unmanaged disclosure path when employees move sensitive data into tools outside organisational control.
• The report shows both a scale problem and a readiness problem, with widespread use but weak visibility and low confidence in secure usage.
• Governance needs to start with discovery and policy enforcement tied to role and data sensitivity, or auditability will remain partial.

Key terms

• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.
• Data Disclosure Boundary: A data disclosure boundary is the point at which sensitive information leaves a controlled environment and enters a system the organisation cannot fully govern. In shadow AI scenarios, that boundary matters as much as account access because the risk is created when data is copied, not only when systems are accessed.
• Policy Enforcement Scope: Policy enforcement scope is the set of infrastructure objects, environments, or workspaces to which a rule is applied. In practice, scope determines where a policy is active, such as a namespace or stack, and whether governance is consistently enforced across the estate.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.