Privacy regulation tightens as AI data use and breach rates rise

At a glance

What this is: Ground Labs' privacy roundup shows regulators tightening controls on AI data use, broker deletion rights and complaint handling while breach rates continue to rise.

Why it matters: It matters because privacy governance is now intertwined with AI pipelines, broker ecosystems and identity verification flows, so practitioners need stronger controls over what personal data is collected, retained and reused.

By the numbers:

• Australia's Office of the Australian Information Commissioner received 1,205 breach notifications in 2025.
• Healthcare accounted for almost 20% of all notifications in Australia.
• Financial services accounted for 13% of all notifications in Australia.

Context

Privacy regulation is moving from policy language to operational controls. The common thread in this roundup is that organisations are increasingly being asked to prove how they collect, classify, retain and delete personal data across AI systems, brokered data flows and complaint processes.

That matters for IAM, NHI and broader security governance because identity data is now feeding model development, verification workflows and downstream data brokerage at scale. When personal data becomes input to machine processing, the privacy control boundary shifts from notice and consent to lifecycle enforcement, minimisation and deletion discipline.

Key questions

Q: What breaks when personal data is collected for AI use without pre-ingestion controls?

A: Privacy controls fail because the organisation no longer knows whether the dataset contains sensitive or unnecessary personal information. That creates downstream risk in model training, retention and re-use, and it makes minimisation impossible to prove after the fact. The control has to move upstream, before the data is absorbed into AI workflows.

Q: Why do data broker deletion requests become difficult to govern at scale?

A: Because deletion is not a single-system event. Brokers must find matching records across multiple systems, resolve identity variations, delete copies and derived data, and stop the same information being repopulated later. The governance problem is record consistency, not just request intake.

Q: How do organisations know whether privacy complaints are being handled properly?

A: They need a named owner, a tracked intake path and a clear acknowledgement timestamp. If complaints are not acknowledged within the required window, the process is failing even if the issue is eventually resolved. Evidence of handling is part of compliance, not an administrative extra.

Q: What is the difference between data minimisation and privacy-preserving techniques in AI governance?

A: Data minimisation limits what data the AI can collect, retain, or process in the first place. Privacy-preserving techniques change how data is used, for example by adding noise, decentralising training, encrypting computations, or replacing identifiers. In practice, minimisation reduces exposure at the source, while privacy-preserving methods help lower risk when some data use remains necessary.

Technical breakdown

Web scraping controls for generative AI

Web scraping for generative AI creates a privacy control problem before model training even starts. If a controller captures content indiscriminately, the organisation inherits purpose limitation, minimisation and data classification obligations for whatever was collected. The key technical issue is not only whether the model can be trained, but whether the dataset contains personal or sensitive information that should have been excluded, replaced with synthetic data or filtered out at collection time. In privacy terms, data governance has to move upstream into the ingestion layer.

Practical implication: build pre-ingestion screening and exclusion checks into web scraping and AI data pipelines before datasets reach model development.

Data broker deletion and record matching

California's DELETE request flow forces brokers to treat deletion as a distributed identity-resolution problem, not a single-record action. A broker must locate matching records across multiple systems, map variants of the same person using available identifiers, delete copies and derived records, and stop deleted data being reintroduced by later collection. This is where privacy governance meets data architecture, because the real control challenge is state consistency across fragmented stores rather than a simple delete button.

Practical implication: inventory where personal data is replicated, then test whether deletion requests actually propagate across primary, copied and derived datasets.

Complaint handling and accountability in privacy operations

The UK complaint rules show that privacy compliance now includes service-level accountability as well as legal principle. Organisations need a clear route for complaints and must acknowledge them within 30 days, which means privacy operations require defined ownership, tracking and response workflows. For identity and security teams, this is a reminder that data subject and complaint handling cannot sit outside the operational control model, especially where identity documents, verification data or model inputs are involved.

Practical implication: assign explicit ownership for complaint intake, acknowledgement and evidence tracking across privacy, legal and security teams.

Threat narrative

Attacker objective: The objective is to exploit retained or poorly governed personal data for theft, reuse or monetisation before privacy controls can limit collection, retention or disclosure.

1. Entry occurs when organisations collect personal data through web scraping, brokered data flows or identity verification processes without adequate upfront filtering.
2. Credential or record abuse follows when copied, derived or retained data is reused across systems after the original privacy purpose has expired.
3. Impact appears as regulatory fines, breach notifications, unlawful data use in AI systems and continued exposure of personal information at scale.

Breaches seen in the wild

• iOS apps leaking hard-coded secrets: Cybernews found 71% of 156,080 iOS apps leak hard-coded secrets, with open cloud storage and Firebase databases exposing user data.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Privacy governance is becoming a data-input control problem, not just a disclosure problem. Once personal data enters AI pipelines, model development, broker ecosystems or verification workflows, traditional notice-based privacy thinking is too late. The operational control now sits before ingestion, where organisations must decide what data may enter at all and how sensitive data is filtered, replaced or excluded. For practitioners, that shifts privacy from paper compliance into enforceable data lifecycle governance.

AI data use is creating a new privacy boundary around collection, not just retention. The article's treatment of generative AI scraping shows that controllers must uphold purpose limitation and minimisation before development begins. That makes web scraping, dataset curation and source approval part of the privacy control stack, especially when datasets may include identity data, location data or other personal attributes. Practitioners should treat training data acquisition as a governed intake process, not an unstructured discovery exercise.

Deletion rights are now an identity-resolution challenge across fragmented records. Broker obligations to find, delete and suppress records across multiple systems expose a recurring governance failure: organisations often know how to receive a request but not how to prove that deletion propagated to copies, derivatives and repopulated data. Persistent personal-data echo: a single request can fail when copied identifiers survive in downstream systems. Practitioners need lifecycle controls that can trace identity-linked data across environments and verify actual removal.

Privacy enforcement is converging with operational accountability. The UK complaint requirement and Australia's breach data both point to the same pattern: regulators expect organisations to run privacy as an operational discipline with clear ownership, response timing and evidence. That is especially relevant where identity verification, AML/CTF checks or AI usage all depend on personal data. For practitioners, privacy governance now needs measurable service delivery as well as policy intent.

What this signals

Persistent personal-data echo: privacy teams should assume that copied and derived records will outlive the original request path unless deletion is tested across every downstream system. That means the real control is not the intake form but the ability to verify suppression in replicated stores, reporting layers and AI data pipelines.

As AI adoption broadens, privacy governance will be judged less by policies and more by whether organisations can prove they prevented unnecessary personal data from entering model development, broker networks and verification workflows. That is a lifecycle control problem, and it belongs in privacy engineering as much as in legal review.

For practitioners

• Implement pre-ingestion data screening for AI pipelines Block or transform personal and sensitive data before it reaches model training, including scraped web content and third-party datasets.
• Test deletion propagation across brokered data stores Map where personal records, copies and derived datasets exist, then verify deletion requests remove each instance and prevent repopulation.
• Define a 30-day complaint acknowledgement workflow Set a named owner, intake channel and evidence trail so privacy complaints are acknowledged within the regulatory window and tracked to closure.
• Separate identity verification from long-term retention Keep the identity proofing necessary for compliance, but remove unnecessary full identity documents once checks are complete.

Key takeaways

• Privacy regulation is converging on the places where personal data is collected, scraped, brokered and reused, not just where it is disclosed.
• Australia's 2025 breach notifications reached 1,205, showing that personal data remains a high-value target for malicious and criminal activity.
• Practitioners need upstream data controls, deletion verification and complaint workflows that produce evidence, not just policy statements.

Key terms

• Purpose limitation: The rule that data should be used only for the specific business purpose allowed by policy and context. In AI environments, this means a dataset may be technically accessible but still inappropriate for a given model, assistant, or agent if the use case exceeds the approved scope.
• Data Minimization: Data minimization is the practice of limiting personal data collection to what is adequate, relevant, and necessary for a specific purpose. It reduces exposure by shrinking what is gathered, transferred, retained, and processed. Strong minimization depends on careful form design, purpose review, and strict collection discipline.
• Deletion Propagation: Deletion propagation is the ability to remove personal data from all systems where it exists, including copies, backups, derivatives and downstream indexes. It matters because a deletion request is only effective when the data disappears from every environment that can still use or expose it.
• Identity Resolution: Identity resolution is the correlation step that determines whether multiple accounts belong to the same person or accountable role. It combines identifiers, context, and system-specific attributes to reduce false splits and missed matches, which is what makes governance outputs dependable rather than approximate.

Deepen your knowledge

Build stronger capability in NHI governance, identity lifecycle control and secrets management through the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for practitioners who need to connect identity controls to broader security and privacy operations.