When a security decision rests on one AI model, an input that the model misreads becomes a way past the control: malware scored as clean, phishing scored as safe, a synthetic face accepted as a real customer. MITRE ATLAS tracks this as AML.T0015 Evade AI Model. It is one of the few ATLAS techniques that sits in three tactics at once, Initial Access, Defense Evasion and Impact, because the same misclassification can open a door, hide an intrusion or break the service the model supports. This guide is about the operational side: where evasion lands in real systems, what the ATLAS case studies show, how to detect it, which of the eight linked ATLAS mitigations to apply first, and how to test. It is for owners of AI-based detection and verification systems, SOC teams that rely on them, and AI red teams. The research background on adversarial examples is in the Adversarial Examples and Evasion Attacks Guide, which this page does not repeat.
Key takeaways
- Evasion is the payoff step. ATLAS describes AML.T0015 as using crafted adversarial data (AML.T0043) or generated deepfakes (AML.T0088) to make a model misjudge input. The crafting happens earlier, often against a replica built with AML.T0005.
- Security models are the main targets. ATLAS cases cover malware detectors, network traffic detectors, botnet domain detectors, email filters, phishing page detectors, face recognition and KYC selfie checks.
- Single-model decisions fail alone. In one ATLAS case the visual similarity model was evaded but other components of the phishing detection system still caught the pages. Layered decisions are the most reliable defence.
- Eight mitigations, three families. ATLAS links model hardening, ensembles and multi-sensor fusion (make the model harder to fool), input restoration, adversarial input detection and deepfake detection (catch bad inputs), and validation and red teaming (prove it before attackers do).
- Identity verification is now an evasion target. Deepfake selfies and injected camera feeds evade face matching and liveness checks; ATLAS records both a research demonstration and a criminal tool sold for this purpose.
How the technique works
ATLAS states that adversaries can craft adversarial data that prevents an AI model from correctly identifying the contents of the data, or generate deepfakes that fool a model expecting authentic data. It notes the technique can be used to evade a downstream task where AI is used, such as AI-based malware detection or network scanning in support of a traditional cyber attack, and that deepfake-based evasion may give initial access to systems that use AI biometric authentication. ATLAS lists the platform as predictive AI and rates maturity as realised. The technique has no sub-techniques; the method variants live under AML.T0043 Craft Adversarial Data.
| ATLAS tactic | What evasion achieves | Example target |
|---|---|---|
| AML.TA0004 Initial Access | Gets an attacker or a fraudulent identity accepted by a gate that uses AI | Face verification at account opening, biometric login, AI-scored onboarding |
| AML.TA0007 Defense Evasion | Hides malicious content or traffic from AI-based detection so a wider intrusion can continue | Malware classifiers, command and control traffic detectors, email and phishing filters |
| AML.TA0011 Impact | Degrades the service the model provides, such as translation, content moderation or image analysis | Machine translation, photo analysis, moderation pipelines |
The adversary's preparation follows a recognisable chain in the case studies: learn about the model (AML.T0063 Discover AI Model Outputs, public research), build or obtain a stand-in (AML.T0005 Replicate AI Model Behavior), craft inputs against it offline (AML.T0043), test them against the real system (AML.T0042 Verify Attack), then use them (AML.T0015). Each step before the last one is a detection opportunity.
Real cases in ATLAS
ATLAS links more case studies to AML.T0015 than to most techniques. Product names are left out here; the ATLAS IDs lead to the full write-ups.
| ATLAS case study | Model evaded | How the input was prepared | Lesson for defenders |
|---|---|---|---|
| AML.CS0003 | Commercial AI malware detector | Verbose logging exposed reputation scoring and a secondary override model; attributes of known good files were fused with malware | Do not expose model internals in logs; an override model is a single point of failure |
| AML.CS0008 | Email protection scoring model | Scores left in email headers were collected to train a copy, which guided mail that scored as safe | Scores visible to senders enable offline evasion |
| AML.CS0000, AML.CS0001 | Deep learning detector for malware traffic; botnet domain generation detector | Proxy models and mutated samples crafted offline | Network detectors need non-AI corroboration |
| AML.CS0032 | Visual similarity model in a phishing page detection system | Brand impersonation pages evaded the visual similarity model | Other system components still detected the pages: layering worked |
| AML.CS0004 | Face recognition in a tax system | Purchased face photos were turned into videos and presented through a virtual camera app; ATLAS reports the scheme collected about $77 million from 2018 | Verify the capture path, not just the image |
| AML.CS0033, AML.CS0034 | Mobile KYC face verification and liveness checks | Live deepfake video injected through a virtual camera; a criminal tool (ProKYC) sold for the purpose, reported by Cato CTRL in October 2024 | Injection attack detection and deepfake detection belong in KYC flows |
| AML.CS0010 | Internal cloud service model (Microsoft AI Red Team exercise) | Model and data obtained through traditional intrusion, adversarial examples built offline, verified through an exposed API | Model files and training data are crown jewels for evasion |
| AML.CS0043 | LLM-based malware analysis tool | Embedded prompt injection aimed at making the analyser not report the binary; ATLAS notes it was not effective against the models Check Point Research tested | LLM analysers inherit prompt injection risk; treat sample text as untrusted |
Detection
Evasion that works produces a wrong answer, so the model's own verdict will not raise the alarm. Detection relies on the stages before the attack, on disagreement between independent signals, and on what happens after the decision.
| Signal | Stage it reveals | Where to look |
|---|---|---|
| Many near-identical submissions with small changes, or verdicts that sit just inside the accept threshold | Probing and verification (AML.T0042) | Model input logs, score distributions per submitter |
| Disagreement between the AI verdict and other controls (sandbox, reputation, rules, human review) | Successful evasion of one layer | Correlation in the SIEM; sample review queues |
| Inputs far outside the training distribution, or with statistical artefacts typical of perturbation | Adversarial input submitted | Input distribution monitors, adversarial input detectors (AML.M0015) |
| Virtual camera drivers, emulators or third-party webcam plugins during verification; device or location data that does not match the documents | Deepfake injection into biometric checks | Device signals in the identity verification platform; FinCEN lists several of these red flags in its alert FIN-2024-Alert004 |
| Accounts verified successfully that later show fraud indicators in clusters | Evasion at onboarding found after the fact | Fraud case data joined to verification records |
| Bulk access to model files, training data or detection logic | Preparation for offline crafting (AML.T0035, AML.T0005) | Registry, storage and repository access logs |
Measuring it
- Adversarial robustness score: detection or match rate on a maintained adversarial test set, tracked per model release against an acceptance threshold.
- Layer disagreement rate: share of decisions where the AI verdict conflicts with an independent control, reviewed weekly.
- Escape rate: confirmed malicious items or fraudulent accounts that the model accepted, per thousand decisions.
- Injection detection coverage: share of biometric sessions checked for virtual camera and emulator signals.
- Red team cadence: days since the last evasion exercise against each production model.
Prevention: ATLAS mitigations mapped
ATLAS release 2026.10 links eight mitigations to AML.T0015. How each mitigation is rolled out across techniques is covered in the MITRE ATLAS Mitigations Guide.
| ATLAS mitigation | What ATLAS says | Practical application |
|---|---|---|
| AML.M0003 Predictive AI Model Hardening | Train models to keep intended performance against adversarial examples, with adversarial training, robust architectures, defensive distillation or certified robustness | Include adversarial samples in retraining; record robustness metrics in the model card |
| AML.M0006 Predictive AI Ensembles | Multiple different models increase robustness | Combine models with different architectures or features so one crafted input does not fool all of them |
| AML.M0009 Predictive AI Multi-Sensor Fusion | A variety of sensors makes compromise harder | Combine camera, depth, device and behaviour signals in biometric and physical systems |
| AML.M0010 Predictive AI Input Restoration | Preprocessing inputs can stop malicious data entering the pipeline | Normalise, re-encode or reduce input precision before inference |
| AML.M0015 Predictive AI Adversarial Input Detection | Detect and block adversarial examples before inference, and monitor query patterns linked to black-box optimisation | Statistical tests, auxiliary detectors and consistency checks in front of the model |
| AML.M0034 Deepfake Detection | Apply deepfake detection to untrusted or user-provided data, especially in biometric verification | Detectors for artefacts and unnatural movement, plus capture path integrity checks |
| AML.M0008 Validate AI Model | Evaluate models against representative adversarial inputs under realistic conditions, with robustness acceptance criteria | A release gate: no promotion to production without passing the adversarial test set |
| AML.M0035 AI Red Team | Run representative digital, multimodal and physical evasion attacks and feed results into robustness, preprocessing, detection, human oversight and monitoring | Scheduled exercises per model; see the AI Red Teaming Guide |
- Keep scores away from the subject: the AML.CS0008 and AML.CS0003 cases show that scores in headers and verbose logs make offline crafting easy. Return verdicts, not numbers, to anything the subject can read.
- Protect model files and training data: white-box access makes crafting far easier. Treat detection models as sensitive assets; see Securing AI Model Weights.
- Use non-AI corroboration for high-impact decisions: sandbox detonation, rules, reputation and human review for edge cases.
- Retrain on escapes: confirmed evasions are the most valuable training data you will get; route them back into the adversarial test set.
Choosing defences by system type
| System | Most likely evasion route | First controls |
|---|---|---|
| AI malware or file classifier | Benign features appended or fused into malicious files | Ensembles with behavioural analysis, no model internals in logs, adversarial test gate |
| Email and phishing detection | Content tuned against a replica built from leaked scores | Verdict-only outputs, layered URL and sender reputation checks |
| Network traffic or domain detector | Mutated traffic or domain names | Corroboration with rules and threat intelligence, input distribution monitoring |
| Face verification, KYC and biometric login | Deepfake or injected camera feed | Deepfake detection, injection attack detection, multi-sensor fusion, step-up checks on risk |
| Image or content moderation | Perturbed images or text | Input restoration, ensembles, human review sampling |
Smaller organisations usually buy these models rather than build them. In that case ask the vendor for adversarial robustness testing evidence, whether scores are exposed and to whom, and what layered checks exist, and run your own red team against the deployed configuration. For biometric flows, see the Biometric Liveness and Presentation Attack Detection Guide and the Deepfake-Resistant Identity Verification Guide.
Where identity fits
AML.T0015 meets identity in two places. First, AI-based identity verification is itself the evaded model when deepfakes defeat face matching at onboarding, and the fraudulent account that results is an identity you now have to find and remove. Second, offline evasion work often starts with stolen access to models and data: in AML.CS0010 the red team used a valid account (AML.T0012) to reach the model file and training data. Our recommendations (NHI Mgmt Group): restrict access to detection model artifacts to named workload identities, alert on bulk reads, and in KYC flows treat injection signals as grounds for step-up verification rather than silent acceptance. See the Identity Proofing and KYC Guide.
Testing and rollout
- Inventory: list every production model that makes or supports a security or trust decision, with its outputs and who can see them.
- Baseline: build an adversarial test set per model from public research, past escapes and red team output, and measure current robustness.
- Quick wins: remove scores and verbose internals from subject-visible outputs; add a non-AI corroborating control for the highest-impact decision.
- Detection in shadow mode: deploy adversarial input or deepfake detection in log-only mode, tune false positives on real traffic, then enforce.
- Release gate: make the adversarial test a promotion criterion (AML.M0008), with the previous model version kept ready as a rollback if the hardened model degrades normal accuracy.
- Red team cycle: schedule AML.M0035 exercises and feed every finding back into steps 2 to 5.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATLAS | AML.T0015; tactics AML.TA0004, AML.TA0007, AML.TA0011 | Technique and tactics |
| MITRE ATLAS related | AML.T0043, AML.T0088, AML.T0005, AML.T0042, AML.T0063 | Preparation steps that precede evasion |
| MITRE ATLAS mitigations | AML.M0003, AML.M0006, AML.M0008, AML.M0009, AML.M0010, AML.M0015, AML.M0034, AML.M0035 | Mitigations ATLAS links to the technique |
| NIST AI 100-2 E2025 | Evasion attacks in the adversarial ML taxonomy | Terminology and mitigation classes; see the Adversarial ML Taxonomy Guide |
| NIST SP 800-53 Rev. 5 | SA-11, CA-8, SI-4, SI-10 | Developer testing, penetration testing, monitoring, input validation |
| NIST CSF 2.0 | ID.RA-01, DE.CM-09, DE.AE-02 | Recording model weaknesses, runtime monitoring, analysis |
Common failure patterns
- One model, one decision: an AI verdict that can accept a file, mail or identity on its own, with no corroborating control.
- Scores and internals exposed: numeric scores in headers or verbose logging that let outsiders map the decision boundary.
- Accuracy-only release testing: models promoted on clean test accuracy with no adversarial test set.
- Trusting the camera: KYC flows that check the image but not whether it came from a real camera on a real device.
- Override models nobody reviews: a secondary allow model that can overrule the main detector becomes the easiest target, as AML.CS0003 shows.
- Escapes not fed back: confirmed evasions handled as incidents but never added to training or test data.
Practitioner checklist
- List every AI model that makes or supports a security or trust decision, with an owner.
- Remove numeric scores and model internals from anything the judged party can see.
- Add at least one independent, non-AI control to each high-impact AI decision.
- Maintain an adversarial test set per model and make it a release gate.
- Deploy adversarial input detection or deepfake detection where inputs come from untrusted parties.
- In biometric flows, check for virtual cameras, emulators and device mismatches, and step up on risk.
- Restrict and monitor access to model files and training data.
- Run an AI red team evasion exercise against each production model at least yearly and after major changes.
Standards and references
- MITRE ATLAS: Evade AI Model
- MITRE ATLAS: Craft Adversarial Data
- MITRE ATLAS: ProKYC: Deepfake Tool for Account Fraud Attacks
- MITRE ATLAS: Predictive AI Adversarial Input Detection
- NIST: AI 100-2 E2025 Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
- FinCEN: Alert FIN-2024-Alert004 on fraud schemes using deepfake media
- Cato CTRL: ProKYC deepfake tool for account fraud attacks
Related NHI Mgmt Group resources: Adversarial Examples and Evasion Attacks Guide · MITRE ATLAS AML.T0031 Erode AI Model Integrity · AI Red Teaming Guide · MITRE ATLAS Guide