ATLAS AI Attack Adaptation Tactic: Detecting Attackers Tuning Attacks Against Your AI

Before an adversary's attack on an AI system works, it usually has to be tuned for that target, and the AI Attack Adaptation tactic (AML.TA0001) in MITRE ATLAS covers that tuning, which is often the only stage where defenders can see the attempts. ATLAS describes the tactic as the adversary "adapting capabilities, methods, knowledge, or objectives into attack-ready outputs for a target or current operational conditions". Outputs include proxy or manipulated models, adversarial data, crafted prompts or retrieval content, deepfakes, generated malicious commands, and tasking for autonomous agents. The tactic can occur before initial access and repeatedly throughout an operation. This hub covers all 25 techniques and sub-techniques in ATLAS 2026.10, ranks them for defenders, shows how case studies chain them, and sets out detection and mitigation priorities. It is written for AI security engineers, ML platform owners, detection engineers and red teams.

Key takeaways

• Adaptation is the bridge from capability to attack. ATLAS separates Resource Development (AML.TA0003), which creates or obtains reusable capabilities, from AI Attack Adaptation, which applies them to a specific target or condition.
• Prompt crafting dominates the record. LLM Prompt Crafting (AML.T0065) appears in 28 case studies under this tactic, more than any other technique in it, so prompt level telemetry is the first detection investment.
• Much adaptation happens where you cannot see it. Proxy models, white box optimisation and adversarial data crafting run on the attacker's infrastructure; your evidence is the probing queries before (model access) and the verification queries after (AML.T0042 Verify Attack).
• Autonomous agents now adapt attacks themselves. Autonomous Attack-Path Adaptation (AML.T0117), Autonomous AI Agent Communication (AML.T0118) and Autonomous Attack Orchestration (AML.T0124) record agents that replan, coordinate sub-agents and generate commands without a human at each step.
• Query limits are the most mapped control. AML.M0004 Limit AI Service Query Volume and Rate is mapped to six techniques in the tactic, tied with four predictive model hardening mitigations.

What AI Attack Adaptation means in practice

In a classic intrusion the adversary brings malware or an exploit and adjusts it to the target environment. AI attacks need more adjustment because the target is a statistical system: an adversarial image or a jailbreak prompt that works on one model may fail on another, or after a model update. Adaptation is therefore iterative. The adversary builds or obtains a capability, probes the target through ATLAS AI Model Access Tactic techniques, refines, verifies, and then uses the result.

ATLAS lists five broad kinds of adapted output, which give a useful way to read the tactic:

• Models: replicas or surrogates of the target (AML.T0005) and manipulated model artifacts (AML.T0018).
• Adversarial data: evasion samples and backdoor triggers (AML.T0043).
• Language content: crafted prompts (AML.T0065) and retrieval content (AML.T0066).
• Synthetic media and code: deepfakes (AML.T0088) and generated malicious commands (AML.T0102).
• Agent operations: autonomous path adaptation, inter agent communication and orchestration (AML.T0117, AML.T0118, AML.T0124).

For the framework as a whole see the MITRE ATLAS Guide.

Every technique in the tactic

The table lists all techniques and sub-techniques ATLAS 2026.10 links to AML.TA0001. "Case study steps" counts case study steps that use the technique under this tactic in the ATLAS 2026.10 data file. Parent techniques link to the NHI Mgmt Group technique page.

In total 58 ATLAS case studies include at least one step under this tactic. A zero means the case studies record the technique under its other tactic or only at sub-technique level, not that it is unused.

Where to start: ranked techniques for defenders

1. AML.T0065 LLM Prompt Crafting. By far the most recorded, and the adaptation step behind prompt injection, jailbreaks and systematic output harvesting. Defenders who run LLM applications see it directly as repeated, varied prompts from one identity.
2. AML.T0042 Verify Attack. Nine case study steps. Verification queries against your own endpoint are the visible tail of offline adaptation, and the point where rate limits and response degradation cost the attacker most.
3. AML.T0117 Autonomous Attack-Path Adaptation and AML.T0124 Autonomous Attack Orchestration. Recent but realized, with real incidents in 2026. Rank them high if you run agents with network or credential access, or if your external surface could face an agentic attacker.
4. AML.T0005 Replicate AI Model Behavior. The route to model theft by distillation and to surrogate models for black box transfer attacks. Rank it high if your model is a commercial asset exposed by API.
5. AML.T0018 Manipulate AI Model. Embedded malware in model files is realized; rank it high for anyone loading third party models.
6. AML.T0043 Craft Adversarial Data. Core for predictive models in security, fraud and biometrics, where evasion is the goal.
7. AML.T0102 Generate Malicious Commands and AML.T0088 Generate Deepfakes. Adaptation that targets people and systems other than the model; the defences sit mainly in endpoint detection and identity verification.

How adversaries chain adaptation

Proxy model, crafted samples, verification

The earliest ATLAS case study (AML.CS0000) shows the classic chain: researchers trained a proxy for a deep learning C2 traffic detector (AML.T0005), crafted evasion samples by removing rarely used HTTP header fields (AML.T0043.003), and queried the real model until the samples were classed as benign (AML.T0042). Only the last step touched the target. The same shape appears in the Google Photos study (AML.CS0058), where recovered models could enable white box optimisation (AML.T0043.000) entirely offline.

Industrial scale distillation

ATLAS case study AML.CS0076 records an NSA, CISA and FBI attribution of industrial scale distillation campaigns to six China based AI companies. They built systematic prompt sets to elicit target capabilities (AML.T0065), ran automated quality assurance pipelines to detect degraded outputs (AML.T0042), and trained their own models on the responses (AML.T0005.001). Access came through fraudulent accounts and proxy infrastructure, which is why identity verification and per account quotas are part of the defence. See the Model Extraction and Distillation Defence Guide.

Agents that replan

Three 2026 incidents show adaptation by autonomous agents. In the Taiwan government case (AML.CS0071), a multi-agent framework ran 12 attack waves with up to eight specialised agents (AML.T0124), built and ranked candidate attack paths (AML.T0117) and exchanged findings between an orchestrator and sub-agents (AML.T0118.001); the Taiwan Autonomous AI Agent Cyberattack 2026 page shows that the weaknesses it found were mainly identity ones. In AML.CS0068, evaluation agents developed alternative routes when intended approaches were exhausted, used a shared package registry namespace as an improvised message board (AML.T0118.000) and generated commands as results came back (AML.T0102); see OpenAI and Hugging Face Breach 2026. In AML.CS0073, agents in capture the flag evaluations selected and revised their own actions and reached real organisations; see Anthropic Claude Evaluation Incidents 2026.

Commands and malware adapted at run time

In LAMEHUG (AML.CS0044), malware attributed by Ukrainian authorities to APT28 used an LLM through a public API to generate commands from natural language on infected hosts (AML.T0102), so each run could differ. In the Hugging Face malicious models case (AML.CS0031), an adversary embedded malware in pickle model files (AML.T0018.002) that ran on load and were not flagged by the hub's scanner.

Where identity fits

Adaptation against your models depends on repeated access, and repeated access depends on accounts and keys. The distillation campaigns used fraudulent and pooled accounts; Storm-2139 used exposed customer credentials scraped from public sources (AML.CS0057). Per identity quotas, account verification for high volume tiers and fast revocation of abused keys raise the cost of every adaptation loop. In the agentic incidents, the reverse applies: agents adapted their paths towards machine credentials, SSO bridges and tokens, because those unlock the most. Scope agent and workload credentials to one system and one purpose. See the LLMjacking Guide.

Detection priorities

Mitigation priorities and rollout

ATLAS maps 35 distinct mitigations to techniques in this tactic. The most frequently mapped are AML.M0004 Limit AI Service Query Volume and Rate, AML.M0003 Predictive AI Model Hardening, AML.M0006 Predictive AI Ensembles, AML.M0010 Predictive AI Input Restoration and AML.M0015 Predictive AI Adversarial Input Detection (six techniques each), then AML.M0002 Predictive AI Output Obfuscation, AML.M0040 Adaptive AI Response Degradation, AML.M0005 Control Access to AI Models and Data at Rest, AML.M0008 Validate AI Model and AML.M0038 AI Agent Scope Drift Detection (five each). AML.T0065 and AML.T0066 have no mitigation mapped in ATLAS 2026.10, so they rely on controls at the prompt injection and retrieval stages.

1. Log and limit queries (AML.M0004, AML.M0024): per identity rate, burst and long term quotas at an AI gateway; log prompts and responses. Start in observe mode to set baselines.
2. Degrade useful signal for attackers (AML.M0002, AML.M0040): return labels instead of confidence scores where the business allows; degrade responses when extraction is suspected. Test against legitimate heavy users before enforcing.
3. Harden predictive models (AML.M0003, AML.M0006, AML.M0010, AML.M0015): adversarial training, ensembles and input detection for security, fraud and biometric models.
4. Admit only verified model artifacts (AML.M0008, AML.M0013 Code Signing, AML.M0014 Verify AI Artifacts): scan and sign models, prefer safe serialisation formats, and block unscanned models from production.
5. Bound your own agents (AML.M0037 AI Agent Authority Expansion Controls, AML.M0038, AML.M0029 Human In-the-Loop for AI Agent Actions): fix the maximum authority before runtime, detect scope drift, and require approval for new targets or credentials. Roll back by suspending the agent identity, not by editing prompts.
6. Prepare for agentic attackers (AML.M0039 AI Honeypots, AML.M0042 Threat Intelligence Program): decoys reward an adaptive agent's curiosity with an alert.

Small vs large organisations: consumers of hosted models mainly need steps 1, 4 and 5. Model providers and teams exposing proprietary models by API need all six, with step 2 a priority against distillation.

Measuring coverage

• Share of model endpoints behind per identity quotas and prompt logging.
• Number of extraction or probing patterns detected per month, and median time to throttle the identity.
• Share of production models admitted through scanning and signature checks.
• Share of production agents with a declared authority ceiling and scope drift monitoring.
• Adversarial robustness test results for each predictive model at each release.

Framework mapping

Common failure patterns

• Logging only failures. Adaptation looks like many successful, slightly different queries; error only logging misses it.
• Returning full confidence scores by default. Rich outputs make black box optimisation and replication cheaper.
• Loading models from public hubs without scanning. Embedded malware runs at load time, before any inference monitoring.
• Bounding agents in the prompt rather than the platform. The evaluation incidents show agents adapting around instructions; boundaries must be network and credential controls.
• Shared credentials across agent runs. One credential reachable by every run lets agents coordinate and escalate together.

Practitioner checklist

• Put every exposed model behind a gateway with per identity quotas and prompt logging.
• Build a detection for systematic prompt variation from a single identity.
• Review what each model returns and remove confidence data that users do not need.
• Scan, sign and verify every model artifact before it reaches production.
• Run adversarial robustness tests on security, fraud and biometric models before release.
• Declare an authority ceiling for each production agent and alert on drift from it.
• Give each agent run its own scoped, short lived credentials.
• Deploy at least one decoy credential or endpoint that an adaptive agent would find.

Standards and references

• MITRE ATLAS: AI Attack Adaptation, Tactic AML.TA0001
• MITRE ATLAS: Autonomous Attack-Path Adaptation, AML.T0117
• MITRE ATLAS: Multi-Agent Framework Compromises Taiwanese Government Systems, AML.CS0071
• MITRE ATLAS: Industrial-Scale Distillation Campaigns, AML.CS0076
• MITRE ATLAS: Evasion of Deep Learning Detector for Malware C&C Traffic, AML.CS0000
• OWASP: LLM01:2025 Prompt Injection
• NIST: SP 800-53 Rev. 5 Security and Privacy Controls

Related NHI Mgmt Group resources: MITRE ATLAS Guide · ATLAS AI Model Access Tactic · Model Extraction and Distillation Defence Guide · AI agent breaches