AML.T0088 Generate Deepfakes: Detecting and Defeating Synthetic Media Attacks

Generative AI has turned convincing fake faces, voices, documents and video from a specialist skill into a cheap, repeatable step in fraud and intrusion, and many verification processes still assume that seeing or hearing someone proves who they are. MITRE ATLAS tracks this as AML.T0088 Generate Deepfakes, in the AI Attack Adaptation tactic (AML.TA0001): adversaries use generative AI to create synthetic imagery, video, audio and text that appear authentic, to impersonate people for phishing or to evade AI systems such as biometric identity verification. This page explains where the technique sits, what the ATLAS case studies show, and how to detect, prevent and respond. It is written for fraud, identity verification, security operations and finance control owners. It deliberately says nothing about how to produce synthetic media. For provenance standards see the AI Content Provenance and C2PA Guide; for the people side see the Deepfake and AI Impersonation Guide.

Key takeaways

• Generation is a preparation step, not the attack. AML.T0088 sits in AI Attack Adaptation: the deepfake is the output that later feeds impersonation, phishing or evasion of a biometric model. Controls work best at the point where the fake is used.
• Two targets: people and models. ATLAS cases cover deepfakes aimed at humans (executives on calls) and at AI systems (face matching and liveness checks in KYC onboarding).
• Scale and real time are what changed. ATLAS notes that generative AI lowers the skill and effort needed and makes real-time manipulation feasible, so one-off checks of media quality no longer hold.
• ATLAS maps two mitigations. AML.M0034 Deepfake Detection and AML.M0009 Predictive AI Multi-Sensor Fusion (for example infrared depth cameras). Both help most where the media is itself the control, as in biometric verification.
• Process beats perception. For payments and access, verification has to use a channel or credential the attacker does not control. Detection is one signal, never the decision.

Where it sits

ATLAS created AML.T0088 on 31 October 2025, last modified it on 27 May 2026, rates it Realized and lists the Predictive AI and Enterprise platforms. The technique description notes that manipulated media is not new, but that generative AI lets adversaries scale operations and run manipulations in real time, often with open-source models built for legitimate use, and in some cases with tooling sold for fraud, such as ProKYC (ATLAS software AML.S0012).

See the AML.T0073 Impersonation page for the impersonation step and its controls; this page focuses on the synthetic media itself and the systems that have to judge it.

Cases in ATLAS and on NHIMG

KYC and onboarding

Cato CTRL describes ProKYC as a tool sold in the criminal underground and customised for cryptocurrency exchanges that verify new users with a government ID and a live camera check, producing a forged document and deepfake video of the same fabricated persona. Cato frames the outcome as new account fraud supporting money laundering and mule accounts. ATLAS also lists Swapface (AML.S0013), which produces real-time face-swapped video delivered to other applications through a virtual camera. The defensive lesson from both: a check that trusts whatever arrives from the "camera" can be fed synthetic frames, so detecting injection matters as much as judging the face.

Executives on a call

In Arup Deepfake Fraud 2024, a Hong Kong finance worker made 15 transfers totalling HK$200 million after a video call with what police said were deepfaked colleagues; it was found when the worker contacted head office directly. No internal systems were compromised: the control that failed was the assumption that a familiar face on a call authorised the request.

Abuse of a generative AI service

Microsoft's Digital Crimes Unit describes Storm-2139 as a network that used exposed customer credentials collected from public sources to access generative AI accounts, altered service capabilities, resold access and gave guidance for producing prohibited imagery. Microsoft filed suit in December 2024, obtained a court order in January 2025 to seize a website central to the operation, and named defendants in an amended complaint in February 2025. For providers, the lesson is that leaked API keys turn your model into someone else's deepfake generator. See AI API key abuse.

Detection

Detection splits by where the synthetic media arrives. In each case the aim is to collect several weak signals rather than trust one detector.

Signals people are told to look for, such as distorted hands, odd eyes or teeth, inconsistent shadows, lag and unnatural movement (all listed in FBI PSA I-120324-PSA), are worth teaching but fade as generators improve. Pair them with process controls that do not depend on spotting a flaw.

Prevention controls

Identity verification and biometrics

1. Require injection attack detection, not only presentation attack detection. Presentation attacks hold a fake up to a camera; injection attacks bypass the camera. Ask your provider for test evidence against both. See the Biometric Liveness and Presentation Attack Detection Guide.
2. Use multi-sensor capture where you control the hardware (AML.M0009). Infrared depth cameras on kiosks and managed devices are harder to fool than a single RGB stream.
3. Layer non-media signals. Device attestation, data checks against authoritative records, velocity rules and links between applications (same face, device or address) catch synthetic identities that pass the visual check.
4. Tune, then route to humans. Cato advises balancing strictness against false rejects and sending suspicious verifications to manual review. Give reviewers the detector outputs, not just the image.
5. Re-verify at risk points. Treat high-risk events after onboarding (payee changes, large withdrawals, credential recovery) as fresh verification moments. See the Deepfake-Resistant Identity Verification Guide.

Payments, access and help desk

1. Out-of-band verification. Call back on a number from the directory, or confirm in an authenticated system; never use contact details supplied in the request.
2. Two-person approval for new payees, bank detail changes and large or unusual transfers.
3. Phishing-resistant authentication for approvals. Approvals happen in a system the requester has signed in to, not on a call.
4. Help desk resets on proof, not presence. A face or voice on a call is not identity proofing for a credential or MFA reset.
5. Agreed verification phrases for executive assistants and finance teams, the workplace version of the FBI's family secret word advice.

Provenance and your own media

Content credentials can show where genuine media came from, which helps you prove your official announcements are real; they cannot prove a file without credentials is fake. Publish official executive video and audio through known channels, consider limiting the volume of executive voice and video that is publicly available, and monitor for impersonating content.

If you run a generative AI service

Protect customer API keys, detect key use from unexpected locations, apply output guardrails for prohibited imagery and voice, and keep abuse reporting and takedown processes ready. ATLAS links Storm-2139 to AML.T0088 because users of the abused service generated the content.

Response

1. Stop the money or the account first. Contact the receiving bank immediately for payments; suspend accounts opened through a suspect verification and freeze their activity.
2. Preserve evidence. Keep call recordings, meeting logs, verification session data (including device and injection signals), documents submitted and payment records.
3. Look for the cluster. Search for other applications or sessions with the same face, document template, device or network indicators.
4. Report. Report fraud to law enforcement (in the US, the FBI's IC3) and request takedown of impersonating content from the hosting platform.
5. Feed it back. Share indicators with your verification provider and update rules and reviewer guidance. See the Deepfake CEO Fraud Response Playbook for the executive fraud case.

Measuring it

Decision guidance

• If you do not run biometric verification: your exposure is mainly people-facing. Spend on payment and reset process controls and training before detection tools.
• If you onboard customers remotely: injection detection, liveness and multi-signal risk scoring are core. Evaluate providers on independent test results; the Deepfake Detection Buyer's Guide covers criteria.
• Small organisations: a call-back rule and second approver for payments addresses the most common loss with no technology spend.
• Large or regulated organisations: combine provider controls, multi-sensor capture where hardware is managed, shared fraud intelligence and post-onboarding re-verification.

Common failure patterns

• Treating a video call as authentication. The Arup case shows recognition of familiar faces can be manufactured.
• Liveness without injection detection. AML.CS0033 shows synthetic frames injected into the capture path can pass both passive and active liveness.
• One detector as the decision. Detection models lag new generators; use them as a signal alongside device, data and behaviour checks.
• Training that relies on visual glitches. Staff learn to trust media that looks clean, which is what improved generators produce.
• Leaked generative AI keys. Storm-2139 used exposed customer credentials to turn a legitimate service into a source of abusive content.

Framework mapping

Practitioner checklist

• Map every process where a face, voice, video or document is accepted as proof of identity.
• Require out-of-band verification and a second approver for payments and bank detail changes.
• Stop help desk credential and MFA resets based on a call alone.
• Confirm your biometric provider tests injection as well as presentation attacks.
• Use multi-sensor capture on managed hardware where feasible.
• Route suspect verifications to trained human reviewers with detector outputs.
• Protect and monitor API keys for any generative AI service you run.
• Rehearse the response: payment recall, evidence preservation, reporting and takedown.

Standards and references

• MITRE ATLAS: Generate Deepfakes (AML.T0088)
• MITRE ATLAS: Deepfake Detection (AML.M0034)
• MITRE ATLAS: ProKYC: Deepfake Tool for Account Fraud Attacks (AML.CS0034)
• MITRE ATLAS: Live Deepfake Image Injection to Evade Mobile KYC Verification (AML.CS0033)
• FBI IC3: Public Service Announcement I-120324-PSA
• Cato Networks: ProKYC deepfake tool for account fraud attacks
• Microsoft On the Issues: Disrupting a global cybercrime network abusing generative AI
• MITRE ATT&CK: Social Engineering: Impersonation (T1684.001)

Related NHI Mgmt Group resources: AI Content Provenance and C2PA Guide, Deepfake and AI Impersonation Guide, Deepfake-Resistant Identity Verification Guide, Executive Deepfake and CEO Fraud Guide, MITRE ATLAS Mitigations Guide.