ATLAS AML.T0043 Craft Adversarial Data: Defences

Before an adversary can slip malware, a phishing page or a face past a machine learning detector, they have to build an input that the model gets wrong, and that preparation is where defenders have the most leverage. MITRE ATLAS tracks this preparation as AML.T0043 Craft Adversarial Data: inputs modified so that they cause the adversary's desired effect in the target model, from misclassification to missed detection to maximised energy use. ATLAS release 2026.10 places the technique in the AI Attack Adaptation tactic (AML.TA0001), applies it to predictive AI, rates it realised and links it to more case studies than almost any other technique. This guide is for owners of ML-based detection and classification systems: security product teams, fraud and abuse teams, and anyone running image, malware, network or document classifiers. It explains the five ways ATLAS says adversarial data is crafted, what each needs from you, and how to cut off that supply. The general mechanics of evasion are in the Adversarial Examples and Evasion Attacks Guide; this page is the ATLAS technique view.

Key takeaways

Crafting depends on access. ATLAS classifies the methods by what the adversary can see: the full model, a query interface, a proxy model they built, or only trial and error.
Real attackers often skip the maths. Manual Modification (AML.T0043.003) is the realised sub-technique, and research on real attacks found that actual attackers use simple tactics rather than gradient computation.
Your API is their test bench. Black-box optimisation needs many queries through an interface the victim may be monitoring, which makes query limits and monitoring a real control.
Protect the model file as well as the endpoint. A model shipped unencrypted inside an app gives an adversary white-box access, the most effective position for crafting.
Layers beat a single model. In a recorded phishing case, manipulated logos evaded the visual similarity classifier but other components of the ensemble still flagged the pages.

How the technique works

ATLAS describes adversarial data as inputs whose modification is usually constrained so a human still perceives them as unmodified, although imperceptibility is not always the goal. It lists five sub-techniques.

Sub-technique	What the adversary needs	ATLAS maturity	Notes from ATLAS
AML.T0043.000 White-Box Optimization	Full access to the target model	Demonstrated	Optimises directly against the model; most effective against that model
AML.T0043.001 Black-Box Optimization	Inference API access (AML.T0040)	Demonstrated	Less effective and needs more inferences than white-box, but much less access; the API may be monitored by the victim
AML.T0043.002 Black-Box Transfer	One or more proxy models, built through AML.T0005 Replicate AI Model Behavior	Demonstrated	White-box attacks on the proxy that are expected to transfer to the target if the proxy is close enough
AML.T0043.003 Manual Modification	Knowledge or guesses about which input features matter	Realised	Trial and error until a working input is verified
AML.T0043.004 Insert Backdoor Trigger	A model already poisoned through AML.T0018.000 Poison AI Model	Demonstrated	Adds a trigger, possibly imperceptible, to inference data so the backdoor fires

Crafting is a staging step. ATLAS notes that with white-box or inference API access, an adversary can confirm the input works before using it (AML.T0042 Verify Attack), then deploy it later to evade the model (AML.T0015 Evade AI Model) or to erode confidence in it over time (see MITRE ATLAS AML.T0031 Erode AI Model Integrity). ATLAS lists the technique for predictive AI only; manipulation of generative models through text is tracked under techniques such as AML.T0051 LLM Prompt Injection and AML.T0054 LLM Jailbreak.

How it shows up in real cases

ATLAS links AML.T0043 and its sub-techniques to more than a dozen case studies. Most are authorised exercises against security products and cloud services; two are recorded incidents. A selection, by method:

ATLAS case study	Type	Method	What happened	Lesson for defenders
AML.CS0032 Attempted Evasion of ML Phishing Webpage Detection System	Incident, December 2022	.003	Logs of a commercial phishing detector showed pages with manipulated brand logos: altered company name style (25 cases), blurred logos (23), cropping (20) and others. They evaded the visual similarity classifier, but other components flagged the pages.	Cheap edits work against one model; independent detectors in an ensemble catch what one misses.
AML.CS0002 VirusTotal Poisoning	Incident	Parent technique	McAfee Advanced Threat Research saw a burst of samples of one ransomware family, generated as mutants of one original with a metamorphic code tool, submitted to a malware-sharing platform.	Watch for unusual volumes of near-duplicate submissions to data you learn from.
AML.CS0003 AI malware detector bypass	Exercise, 2019	.003	Researchers reverse-engineered a detector's reliance on string features and appended content from a benign program to malicious files; they reported 83.59 per cent of 384 malicious files evaded detection.	Know which features your model leans on and test against inputs that target them.
AML.CS0011 Edge AI evasion	Exercise, 2020	.001	A red team built an automated system that kept perturbing an image until the model misclassified it, with changes invisible to people.	Black-box optimisation is a high-volume query pattern you can detect.
AML.CS0012 Face identification evasion via physical countermeasures	Exercise, 2020	.000 on a proxy	MITRE's AI Red Team optimised a physical patch on a proxy model to cause targeted misclassification by a commercial service.	Physical-world inputs need adversarial testing too.
AML.CS0014 Confusing antimalware neural networks	Exercise, 2021	.002	A research team built adversarial files against a proxy model and transferred them to a cloud-based detector.	Limiting what your outputs reveal makes good proxies harder to build.
AML.CS0058 On-device model extraction	Exercise, 2025	.000	Researchers recovered TensorFlow Lite models stored unencrypted in a mobile app's assets, which could enable white-box adversarial example generation.	Shipping a model to devices may hand over white-box access.
AML.CS0013 Backdoor attack on mobile app models	Exercise, 2021	.004	Researchers injected a neural payload into on-device models; a physical trigger captured by the camera activated it.	Model integrity checks prevent the backdoor that a trigger depends on.

The pattern matches the research paper "Real Attackers Don't Compute Gradients" (December 2022), whose authors argue that actual attackers use simple tactics to subvert ML-driven systems. Defences tuned only to gradient-based perturbations miss most of what arrives in production.

Deny access and feedback

Every crafting method needs either the model, a query channel or a good proxy. ATLAS mitigations map onto each.

What the adversary needs	ATLAS mitigation	Practical control
The model itself (.000)	AML.M0005 Control Access to AI Models and Data at Rest; AML.M0017 AI Model Distribution Methods	Restrict access to registries and training data; serve sensitive models from the cloud rather than shipping them to devices, or protect on-device models where they must ship. See Securing AI Model Weights.
Many queries (.001, .003)	AML.M0004 Limit AI Service Query Volume and Rate; AML.M0019 Control Access to AI Models and Data in Production	Authenticate callers, limit query volume per identity, and monitor production queries for usage policy violations.
Informative outputs (.001, .002)	AML.M0002 Predictive AI Output Obfuscation	Return labels or coarse bands instead of raw confidence scores to external callers.
Stable feedback during an attack	AML.M0040 Adaptive AI Response Degradation	ATLAS describes degrading responses during suspected adversarial activity to limit the feedback used to optimise inputs.
A poisoned model for a trigger (.004)	AML.M0008 Validate AI Model	Validate models before deployment; see the AI Data Poisoning and Backdoors Guide.

Make crafted inputs fail

When access cannot be denied, as with a public detection service, the remaining ATLAS mitigations make crafted inputs less effective.

AML.M0003 Predictive AI Model Hardening: train for robustness, for example with adversarial training, and include manual-modification styles seen in your own logs, not only gradient-generated examples. The Adversarial Robustness Toolbox Guide covers tooling.
AML.M0006 Predictive AI Ensembles: ATLAS says ensembles increase the difficulty of crafting effective adversarial data. Use models that rely on different features, as the phishing incident showed.
AML.M0010 Predictive AI Input Restoration: preprocessing such as denoising, compression or resampling to disrupt perturbations before inference. ATLAS advises evaluating restoration methods against adaptive adversaries that account for the preprocessing step, and you should also measure the effect on accuracy for clean inputs.
AML.M0015 Predictive AI Adversarial Input Detection: detect and block digital or physical adversarial examples before inference, and monitor for input characteristics or query patterns associated with crafting.
Non-ML backstops: keep signatures, reputation, rules or human review alongside the model for high-impact decisions, so one evaded model is not one missed detection.

Choosing controls

Deployment	Most likely crafting method	Priority controls
Public scoring API (fraud, content, malware verdicts)	.001 black-box optimisation, .002 transfer	Authentication, per-identity query limits, coarse outputs, query pattern monitoring
Model shipped in mobile or edge apps	.000 white-box, after extracting the model	Avoid shipping sensitive models, protect assets, server-side confirmation of high-impact results
Inline detector the adversary cannot query directly (email, web, endpoint)	.003 manual modification, .002 transfer from a proxy	Ensembles with diverse features, retraining on observed evasions, non-ML backstops
Physical-world sensing (cameras, biometrics)	.000 or .002 producing physical patches	Physical adversarial testing, multi-sensor confirmation, liveness and human review
Models trained on third-party data	.004 trigger for an implanted backdoor	Data provenance, model validation, trigger scanning in evaluation

A small team with one exposed model gets most of the benefit from authentication, query limits and coarse outputs. Security vendors and large platforms whose models are a known target need continuous adversarial testing and a retraining loop fed by evasions seen in the wild.

How to implement

Classify exposure: for each model, record who can query it, what it returns, and whether the model file leaves your control.
Reduce feedback: remove raw scores from external responses where the business does not need them, and test that legitimate integrations still work.
Add query controls: authentication and per-identity limits in monitor-only mode first, then enforce once the baseline is clear.
Build a crafted-input test set: combine library-generated adversarial examples with manual modifications based on real evasions; run it as a release gate for every retrain.
Add diversity: introduce a second model or non-ML check on high-impact decisions and alert on disagreement.
Close the loop: route confirmed evasions from incident handling back into training and the test set.
Roll back: keep the previous model version deployable in case a hardened model loses too much clean accuracy.

Where identity fits

Black-box crafting runs on API access, and ATLAS warns that adversaries evade query limits through distributed requests, account rotation or stolen credentials. Tie every query to a known application or customer identity, use separate keys per integration so abuse can be cut off narrowly, and watch for many new accounts or keys producing similar query sequences. See AML.T0012 Valid Accounts for the account side of AI attacks.

Detection and measurement

Near-duplicate sequences: many queries from one identity that differ by small changes to one input, the signature of black-box optimisation.
Boundary probing: clusters of inputs whose scores sit close to the decision threshold.
Disagreement: cases where ensemble members or a non-ML check disagree with the primary model.
Post-incident evasions: items later confirmed malicious that the model allowed, reviewed for shared modification styles.
Submission bursts: unusual volumes of near-identical samples entering data you learn from, as in the ransomware mutant case.

AML.M0024 AI Telemetry Logging and Monitoring provides the input and output records these checks need.

Metrics

Adversarial test pass rate: detection rate on the crafted-input test set per model version.
Clean accuracy cost: change in accuracy on normal data after hardening or input restoration.
Exposed-score share: share of external endpoints that return raw confidence scores.
Evasion feedback time: time from a confirmed evasion to its inclusion in training or the test set.
Probing alerts: number of identities flagged for near-duplicate query sequences, and share confirmed.

Framework mapping

Framework	Reference	Relevance
NIST AI 100-2 E2025	Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations	Evasion attacks and attacker knowledge and capability; see the Adversarial ML Taxonomy Guide
NIST SP 800-53 Rev 5	SI-4 System Monitoring; SI-10 Information Input Validation	Query monitoring and input checks before inference
NIST SP 800-53 Rev 5	AC-6 Least Privilege; CA-8 Penetration Testing	Restrict model access; adversarial testing
NIST CSF 2.0	DE.CM-09; DE.AE-02	Runtime monitoring and analysis of adverse events
MITRE ATLAS	AML.M0035 AI Red Team	Recurring threat-informed testing; see the AI Red Teaming Guide

Common failure patterns

Testing only with gradient attacks: robustness looks good in the lab while simple manual edits succeed in production.
Raw scores to anyone: external callers get full confidence values, giving black-box optimisation a clear signal.
Single-model verdicts: a high-impact allow or block decision rests on one classifier.
Models shipped in the clear: on-device model files that anyone can extract for white-box crafting.
Stale models: evasions seen in the wild never reach retraining, so the same trick keeps working.

Practitioner checklist

Record each model's query exposure, outputs and distribution.
Authenticate and rate limit every external inference endpoint per identity.
Return coarse outputs instead of raw scores where possible.
Keep sensitive models off client devices or protect them where they must ship.
Maintain a crafted-input test set that includes manual modifications and run it at every release.
Use diverse ensembles or non-ML checks for high-impact decisions.
Alert on near-duplicate query sequences and boundary probing.
Feed confirmed evasions back into training within a set time.

AML.T0043 Craft Adversarial Data: Deny Attackers the Access and Feedback They Need