Data Retention and Secure Deletion Guide: Schedules and Sanitisation

Data that should have been deleted years ago is breach scope, legal discovery scope and regulatory exposure, yet many organisations can neither say how long they keep it nor prove it is gone. Retention and deletion sit between privacy, records management, legal and security, so each team assumes another owns them. This guide covers the full lifecycle: building a retention schedule, enforcing it automatically across live systems, backups, endpoints and AI services, handling legal holds, and disposing of data and media securely under NIST SP 800-88 Rev. 2. It is written for security, privacy and infrastructure practitioners who have to turn a retention policy into controls that work and evidence that stands up.

Key takeaways

• Retention is a security control. Every record kept past its purpose enlarges the blast radius of the next breach and the cost of the next lawsuit.
• The schedule is only as good as its boundary. Replicas, backups, logs, SaaS exports, endpoints and AI caches must follow the same rule or the deletion is cosmetic.
• Legal hold overrides deletion, always. Automated deletion needs a hold check built in, or it becomes spoliation.
• Sanitisation is chosen by data sensitivity, not media type. NIST SP 800-88 Rev. 2 uses Clear, Purge and Destroy; degaussing does nothing to SSDs.
• Crypto shredding works only with clean key hygiene. If plaintext copies or escrowed keys exist, destroying the key does not destroy the data.

Why retain, why delete: the obligations

A data retention policy sets how long each class of information is kept and what happens at the end. Two forces pull in opposite directions.

• Compliance-driven data retention: laws, regulators and contracts require some records to be kept for a minimum period (tax, payroll, financial trading records, clinical records, security logs). Broker-dealers subject to SEC Rule 17a-4 need records that cannot be altered or deleted early, which is why cloud platforms offer write-once controls assessed against it.
• Data retention and minimisation: privacy law requires deletion once the purpose is spent. UK and EU GDPR Article 5(1)(e) says personal data must be "kept in a form which permits identification of data subjects for no longer than is necessary". The ICO adds that "the UK GDPR does not dictate how long you should keep personal data", and that you should not keep it "just in case".
• Erasure on request: GDPR Article 17 gives individuals a right to erasure, with exceptions including the establishment, exercise or defence of legal claims. A schedule that already deletes on time makes erasure requests routine. See the DSAR Guide: Handling Data Subject Requests.
• Security exposure: old data rarely gets the controls current data does. In the GoTo breach, attackers took encrypted backups together with an encryption key, so archived data became part of the incident.

Retention periods themselves are a legal decision. Security's job is to make whatever period is chosen enforceable, measurable and provable.

Building the retention schedule

The ICO describes retention schedules as lists of "the types of record or information you hold" with how long you intend to keep them, and expects organisations to "establish and document standard retention periods for different categories of information". Under GDPR Article 30, records of processing should include envisaged time limits for erasure where possible. A workable schedule has these columns:

An employee data retention policy is usually the first schedule to get right because HR data is sensitive, scattered (applicant tracking, payroll, occupational health, monitoring logs) and governed by several laws at once. Treat unsuccessful candidates, leavers and current staff as separate classes with separate triggers. Data that has been truly anonymised falls outside storage limitation; pseudonymised data, as the ICO notes, "will usually still permit identification" and does not.

The data retention boundary

The data retention boundary is every place a record class exists, including copies nobody planned. Deletion that stops at the primary database is not deletion. The ICO's test is to "put the data beyond use", including backups where appropriate, and it notes that offline storage still needs a justification.

Map the boundary from the data inventory. See the Data Mapping and ROPA Guide and the Data Classification Guide.

Data retention automation and operational enforcement

Operational data retention enforcement means the platform deletes on schedule without anyone remembering to. Manual clean-ups are the most common reason retention fails audits.

• Microsoft Purview: retention policies apply at site or mailbox level and retention labels at item level, with labels travelling with content inside the tenant. Microsoft documents four principles for conflicts: retention wins over deletion, the longest retention period wins, explicit (label) wins over implicit (policy) for deletions, and the shortest deletion period wins. Retained copies sit in the Preservation Hold library, Recoverable Items or the SubstrateHolds folder, so they remain discoverable until the period ends.
• Preservation Lock and regulatory records: where a regulation such as SEC Rule 17a-4 requires that retention cannot be weakened, Preservation Lock stops anyone, administrators included, from turning a policy off. Use it deliberately: a locked policy cannot be undone.
• Object storage: lifecycle rules expire objects by age. Amazon S3 Object Lock adds write-once-read-many protection: in compliance mode no user, including the root user, can delete or shorten retention; governance mode can be bypassed only with s3:BypassGovernanceRetention. Test periods in governance mode first.
• Databases and applications: scheduled jobs that delete or anonymise by trigger date, with row counts logged. Build deletion into the data model (a retention class per table or record) rather than bolting it on.
• Review before deletion: the ICO accepts automated systems that flag records for review or delete them after a set period. Use review only for high-impact classes; review queues for everything never empty.

Legal hold and litigation hold

A legal hold (litigation hold) suspends deletion for data relevant to actual or reasonably anticipated litigation, investigation or audit. In US federal courts, Federal Rule of Civil Procedure 37(e) lets a court order curative measures where electronically stored information that should have been preserved is lost because a party failed to take reasonable steps, and allows adverse inferences, dismissal or default judgment where the party intended to deprive the other side of it.

1. Trigger: legal issues the hold notice naming custodians, systems, date range and record classes.
2. Apply technically: eDiscovery holds in Microsoft 365 take precedence over every retention setting; S3 legal holds have no expiry and stay until removed by a principal with s3:PutObjectLegalHold; suspend backup rotation for affected sets where needed.
3. Pause automation: every deletion job checks a hold register before it runs, including offboarding wipes of a custodian's devices and mailbox.
4. Release: legal releases the hold in writing and normal retention resumes, which may mean immediate deletion of overdue data.

Microsoft's guidance is explicit: "don't use eDiscovery holds for long-term data lifecycle management." Holds are short-term and user-specific; retention is long-term and content-based. Treat the right to place or remove holds as a privileged permission.

Secure deletion and media sanitisation

Secure deletion in a live system removes data so that normal and forensic recovery are infeasible; media sanitisation treats the storage itself before reuse or disposal. NIST SP 800-88 Rev. 2, published September 2025 and superseding Rev. 1, defines media sanitisation as "a process that renders access to target data on the media infeasible for a given level of effort". Rev. 2 focuses on the sanitisation programme and points to IEEE 2883 for technology-specific techniques.

• Decide by sensitivity: NIST states the decision "is based on the confidentiality of the information rather than the type of media", together with whether the media will be reused and whether it leaves organisational control.
• Degaussing: NIST says it "should not be used for non-magnetic ISM (e.g., flash storage, such as SSDs)" and is "not considered an approved destroy sanitization technique". Older degaussers may lack the field strength for modern high-coercivity drives. A degausser run on an SSD can report success while sanitising nothing.
• SSDs and flash: overwriting does not reach over-provisioned or remapped blocks. Use the drive's sanitize command or cryptographic erase, then verify.
• Physical destruction: the NCSC asks for particles of 6mm or less, verified, and recommends erasing data first.
• Everything with storage: the NCSC's rule is that any device that "might" contain storage media is sanitised, including printers, network appliances and phones, and it warns that "Simply hitting the 'Delete' key isn't enough."

Crypto shredding and cryptographic erasure

Crypto shredding deletes data by destroying the key that encrypts it. NIST calls it cryptographic erase (CE) and describes it as "based on the sanitization of keys used to encrypt data or to prevent access to the keys". It is the only practical way to delete data spread across cloud storage, replicas and backups that you cannot physically reach.

• Preconditions: NIST requires that the data was never stored in plaintext on the media, encryption of at least 128-bit security strength, and (for US federal agencies) FIPS 140 validated modules. CE "should not be trusted on ISM that have been backed up or escrowed" unless key handling outside the media is understood.
• Key granularity: one key per tenant, customer, dataset or retention class makes targeted shredding possible. One key for everything makes it impossible.
• Cloud key deletion: AWS KMS enforces a 7 to 30 day waiting period (default 30) in the "Pending deletion" state, after which data encrypted under the key "becomes unrecoverable". AWS advises disabling a key first if unsure and alarming on any use during the waiting period. Deleting imported key material is reversible if you still hold it, so destroy your own copy too.
• Self-encrypting drives and devices: the NCSC accepts that a manufacturer factory reset of an encrypted device, which deletes the keys, "will provide a satisfactory level of assurance" for most devices.

See the Cryptographic Key Management Guide for key hierarchy design.

Data disposal, chain of custody and certificates of destruction

Sensitive data disposal fails most often between the server room and the destruction facility. Track every asset from decommissioning to destruction, and require a certificate of destruction (NIST calls it a certificate of sanitization) per device. NIST SP 800-88 Rev. 2 lists the minimum fields: manufacturer, model and serial number; organisational property number; media type and source; sanitisation method and technique; tool and version; verification method; and the name, title, date, location, contact details and signature of the verifier and validator. A batch certificate with no serial numbers is not evidence. For outsourced data destruction, the NCSC points to providers certified under its CAS(S) sanitisation assurance scheme or working to recognised standards, and asks cloud providers for assurance that remnants stay protected until overwritten.

AI data retention and zero data retention

An AI data retention policy extends the schedule to prompts, outputs, uploaded files, fine-tuning data, vector stores and agent memory. AI services retain data in more places than the main API suggests.

• Zero data retention is a configuration, not a checkbox: Google Cloud's documentation says that "to achieve zero data retention, customers must take specific actions within each of these areas": abuse-monitoring prompt logging, request-response logging, stored interactions, in-memory caching (deleted "no later than 24 hours after it's written"), grounding features that keep data for days and cannot be disabled, and stored agent sessions. Other model providers have similar layers; check each one against the contract.
• AI agent data retention: agents write conversation history, tool outputs and long-term memory. Give each memory store a retention class and TTL, delete source records and their embeddings together, and include agent stores in erasure requests. See the AI Agent Memory Security Guide and the AI and Privacy Guide.
• Deletion authority for agents: an agent token that can delete data can delete the wrong data. The Replit AI agent database deletion shows why destructive rights should be scoped, approved and recoverable.

Endpoint data retention

Endpoint data retention covers local downloads, synced folders, browser caches, offline mailboxes and mobile apps. Redirect known folders to managed cloud storage so retention applies centrally, set cache limits for offline mail, and use device management actions at offboarding: a selective retire removes company data from personal devices, while a full wipe resets corporate devices before reissue. Reissued and returned laptops still need sanitisation under the media rules above. See the Lost or Stolen Device Response Checklist.

Decision guidance: small and large organisations

• Small organisations: the ICO accepts that occasional, low-risk processing may not need a documented policy, but a one-page schedule covering HR, customer and finance records is cheap. Use built-in SaaS retention settings, factory reset encrypted devices before reuse, and buy certified destruction for everything else.
• Large organisations: need a records owner per class, a hold register integrated with deletion jobs, crypto shredding by tenant or dataset, and evidence pipelines that feed audit.
• Delete versus anonymise: anonymise when aggregate value remains and re-identification risk is assessed; delete otherwise.
• Purge versus destroy: purge when media has resale or reuse value and verification is possible; destroy failed, unverifiable or highest-sensitivity media.

Implementation: a phased rollout

1. Inventory and classify: list record classes, systems and copies; agree owners.
2. Agree the schedule: legal sets periods and triggers; record them in the ROPA.
3. Stand up the hold register: before enabling any automated deletion.
4. Pilot automation in report-only mode: run deletion jobs or retention policies that only report what they would remove; compare against owner expectations; use governance mode rather than compliance mode while testing.
5. Enable deletion by class: start with low-risk, high-volume data; keep backups of deleted sets for a short, documented grace period so mistakes can be reversed.
6. Extend the boundary: backups, logs, endpoints, AI services and processors.
7. Formalise disposal: sanitisation procedure, certified providers, per-asset certificates.

Detection and measurement

• Log: retention policy and label changes, hold placement and release, bulk deletions, KMS key deletion scheduling (CloudTrail records both scheduling and deletion), and any use of a key in "Pending deletion".
• Alert: policy disabled or shortened, hold removed outside legal workflow, governance-mode bypass, deletion volume far above baseline (which can also indicate a destructive attacker).

Metrics:

• Percentage of record classes with an approved period, trigger and owner.
• Percentage of in-scope systems with automated retention enforced.
• Volume of data past its retention date (should trend to zero).
• Days from asset decommissioning to certificate of destruction.
• Assets disposed with a matching per-serial certificate (target 100%).
• Deletion jobs blocked correctly by active holds versus holds missed.

Framework mapping

Common failure patterns

• Schedule without enforcement: a policy document that no system implements, discovered when a breach exposes decade-old records.
• Backups outside the schedule: snapshots kept on default settings long after the source data was deleted. The LastPass breach showed that cloud backups are a target in their own right.
• Automated deletion that ignores holds: a routine purge destroys evidence after litigation is anticipated.
• Degaussing SSDs: certificates say "sanitised" while every flash cell is intact.
• Crypto shredding with leaky keys: keys escrowed, exported or shared across datasets, so destroying one copy changes nothing.
• Batch certificates: destruction evidence that lists a weight or count, not serial numbers.

Practitioner checklist

• Publish a retention schedule with class, basis, trigger, period, end-of-life action, systems and owner.
• Map the retention boundary for each class, including backups, logs, endpoints, processors and AI services.
• Integrate a legal hold register with every automated deletion job and offboarding workflow.
• Enforce retention natively (Purview policies and labels, lifecycle rules, Object Lock) and test in report-only or governance mode first.
• Choose Clear, Purge or Destroy by data sensitivity and destination, per NIST SP 800-88 Rev. 2; never degauss flash media.
• Design key granularity so crypto shredding can target a tenant, dataset or retention class.
• Require per-serial certificates of sanitisation with the NIST minimum fields.
• Configure AI services and agent memory stores for the retention the policy promises, including caches and logs.
• Alert on retention policy changes, hold removals and key deletion scheduling.

Standards and references

• NIST: SP 800-88 Rev. 2, Guidelines for Media Sanitization
• ICO: Principle (e): Storage limitation
• NCSC: Secure sanitisation and disposal of storage media
• Cornell LII: Federal Rules of Civil Procedure Rule 37
• Microsoft Learn: Learn about retention policies and retention labels
• AWS: Locking objects with Object Lock
• AWS: Delete an AWS KMS key
• Google Cloud: Zero data retention for generative AI

Related NHI Mgmt Group resources: Data Security Guide · GDPR Compliance Guide for Security Teams · Backup Security and Immutable Backup Guide · Log Management and Retention Guide · Encryption Guide