Data masking and tokenisation tools promise to let teams use sensitive data without exposing it, but the products sold under these labels range from simple display masks to cryptographic services that change a payment card environment's audit scope. Buying the wrong kind leaves real data in test systems, or a token vault that becomes the single most valuable target in the estate. This vendor-neutral guide helps security, data platform and privacy teams define requirements, compare capabilities, question vendors and run a proof of concept. It assumes you already know where your sensitive data is; for the techniques themselves, see the Tokenisation and Data Masking Guide.
Key takeaways
- Match the technique to the use case. Static masking for non-production copies, dynamic masking for role-based views, vaulted or vaultless tokenisation for regulated identifiers in production, and statistical de-identification for data released for analysis.
- Pseudonymised is not anonymous. The UK ICO states that pseudonymised data remains personal data for anyone holding the additional information needed to re-identify it, so tokenised personal data still needs data protection controls.
- The detokenisation path is the risk. Keys, mapping tables and detokenisation APIs must be separated from tokenised data, tightly authorised and logged; they are non-human access paths in their own right.
- Scope reduction is earned, not bought. The PCI SSC tokenisation guidance is clear that tokenisation does not remove the need to maintain and validate PCI DSS compliance, and that the tokenisation system itself is always in scope.
- Check the cryptography's standing. NIST's second public draft of SP 800-38G Rev. 1 (February 2025) specifies only FF1 for format-preserving encryption and no longer specifies FF3; ask vendors what they use.
Define requirements
- Use cases: test and development data, analytics and data science, outsourced support views, payment card storage, data sharing with partners, training data for AI models. Each favours a different technique.
- Data types and formats: card numbers, national identifiers, names, addresses, dates of birth, free text, documents and images. Note which fields must keep their format, length or check digits for downstream applications.
- Reversibility: does any process legitimately need the original value back? If not, prefer irreversible techniques and avoid holding a vault at all.
- Referential integrity: must the same input always produce the same output across tables and systems so joins still work (deterministic), or must outputs be unlinkable (randomised)?
- Platforms: databases, cloud warehouses, lakehouses, SaaS applications, streaming pipelines and file stores in scope, and whether protection must apply at ingestion, at rest or at query time.
- Compliance drivers: PCI DSS scope reduction, UK GDPR and EU GDPR pseudonymisation (Article 4(5)) as a security measure, health data rules, data residency. See the PCI DSS 4.0 Guide and the Privacy-Enhancing Technologies Guide.
- Performance and availability: tokenisation in the transaction path becomes a dependency; set latency and uptime requirements before evaluating.
Capability areas
| Area | What it does | What to look for |
|---|---|---|
| Discovery and classification | Finds sensitive fields to protect | Coverage of your platforms and formats; integration with existing catalogues and labels. See the Data Classification Guide |
| Static masking | Irreversibly replaces values in a copy, typically for non-production | Consistent masking across databases, realistic substitutes, handling of free text, repeatable pipelines |
| Dynamic masking | Masks values in query results by role; stored data unchanged | Policy granularity (column, row, role, attribute), behaviour for admin roles, resistance to inference through filters |
| Vaulted tokenisation | Random token mapped to the original in a secured vault | Vault protection, high availability, replication, token format options, detokenisation authorisation |
| Vaultless tokenisation and format-preserving encryption | Tokens derived cryptographically from the value and a key | Algorithm and its standing, key management, key rotation impact on existing tokens |
| Statistical de-identification | Generalisation, suppression, noise, synthetic data for analysis or release | Re-identification risk measurement, documentation for a disclosure review |
| Key and secret management | Protects keys, salts and mapping tables | HSM or external key manager support, separation from data, rotation. See the Cryptographic Key Management Guide |
| Access control and audit | Decides who or what may unmask or detokenise | Per-identity authorisation, rate limits, full audit logs, integration with your IdP and SIEM |
NIST SP 800-53 Rev. 5 covers this ground in SI-19 (De-identification) and SC-28 (Protection of Information at Rest); NIST SP 800-188 adds governance guidance, such as a disclosure review board and re-identification studies, for organisations releasing de-identified data, and warns that tools which only mask personal information may not be sufficient for de-identification.
Evaluation criteria
| Criterion | Strong answer | Weak answer |
|---|---|---|
| Token generation | Random tokens with no mathematical link to the original, or a documented, current NIST-approved mode for FPE | Proprietary or undisclosed algorithm; FF3 or FF3-1 without a migration plan |
| Token distinguishability | Tokens can be told apart from real card numbers, as PCI SSC guidance advises | Tokens pass Luhn checks and cannot be distinguished from PANs |
| Separation of additional information | Vault, keys and salts held separately with their own access control, as ICO guidance describes | Mapping table in the same database as the tokenised data |
| Detokenisation control | Per-application identity, field-level scope, rate limits, logged requests | One shared API key with full detokenise rights |
| Dynamic masking bypass | Documented behaviour for admins and inference queries; masking combined with access control | Masking presented as a security boundary against privileged users |
| Static masking quality | Consistent across systems, preserves test value, covers free text and attachments | Column-by-column scripts that miss copies and logs |
| Operational fit | Measured latency, high availability, clear failure mode, automation APIs | Single-region vault in the payment path |
| Evidence | Reports for auditors on what is protected, by which method, and who detokenised | No reporting beyond a configuration export |
Questions to ask vendors
- Which tokenisation and format-preserving encryption methods do you support, and how are you responding to NIST's draft revision of SP 800-38G, which drops FF3?
- Where are the vault, keys and salts held, who at your company can reach them, and can we hold the keys in our own key manager or HSM?
- How does each calling application authenticate to detokenise, how are its credentials rotated, and can we limit it to specific fields and volumes?
- How can a token be distinguished from a real card number, and do you share that method for our PCI DSS assessment?
- What happens to existing tokens when a key is rotated or compromised?
- Which database and warehouse administrator roles can see unmasked data under dynamic masking, and how do you prevent inference through filtered queries?
- How does static masking keep referential integrity across databases, and how does it handle free text, logs and backups?
- What is the measured latency and availability of the tokenisation service, and how does the calling application behave if it is unavailable?
- What audit evidence can you produce for a regulator or a PCI QSA?
Red flags
- Claims that tokenisation takes systems "out of PCI scope" without explaining segmentation and the conditions in the PCI SSC guidance.
- Claims that pseudonymised or tokenised personal data is anonymous and outside data protection law.
- Undisclosed or proprietary cryptography, or reliance on FF3 or FF3-1 with no plan.
- Detokenisation through a single shared credential, or no per-request audit trail.
- Keys or mapping tables stored alongside the protected data. The GoTo breach shows the result: attackers took encrypted backups together with an encryption key for part of them.
- Dynamic masking sold as protection against database administrators or owners. Microsoft's Azure SQL documentation, for example, states that administrators and db_owner members are not masked.
- Static masking that depends on hand-written scripts per table, with no way to prove coverage.
Proof of concept
- Pick two real use cases, for example a non-production copy of a customer database and card number tokenisation in one application, and define success criteria in advance.
- Run discovery on a representative dataset and compare results with your own classification; record missed and false positive fields.
- Produce a masked copy and have testers confirm it still works; then try to re-identify a sample of records using other data you hold, applying the ICO's motivated intruder approach.
- Tokenise and detokenise through the API under load; measure latency, and test behaviour when the service is unavailable.
- Try to detokenise with an unauthorised identity, exceed rate limits, and query masked columns as an administrator; confirm each is blocked or logged as expected.
- Rotate a key and confirm what happens to existing tokens and applications.
- Export the audit evidence and review it with your PCI QSA or DPO before deciding.
How NHI Mgmt Group can help
We provide independent requirements, RFP and evaluation support for data security programmes, including the non-human identities that call tokenisation and detokenisation services. Browse vendors in our products directory or contact us.
Related NHI Mgmt Group resources: Tokenisation and Data Masking Guide · Data Access Governance Guide · DSPM Buyer's Guide · Database Security Guide · Encryption Buyer's Guide