Attackers now build capabilities aimed at AI systems, and increasingly build them with AI agents, before they ever touch your network. MITRE ATLAS records this as AML.T0017 Develop Capabilities, a Resource Development (AML.TA0003) technique with three sub-techniques: adversarial AI attacks, autonomous exploit development and AI agent tools. Like its ATT&CK counterpart, T1587 Develop Capabilities, most of this work happens outside your view. This guide covers only the AI-specific angle: what each kind of capability looks like, the few places defenders can see it being built or tested, and how to make your estate a poor target for it. It is for threat intelligence, detection and AI platform teams.
Key takeaways
- Mostly invisible, not irrelevant. MITRE ATT&CK says develop capabilities cannot easily be mitigated with preventive controls because it happens outside enterprise defences, so the value lies in anticipating what is being built and hardening against it.
- Three AI-specific capability types. AML.T0017.000 Adversarial AI Attacks, AML.T0017.001 Autonomous Exploit Development and AML.T0017.002 AI Agent Tools each leave different traces and need different defences.
- Development often needs your systems. Adversarial attacks are tuned by querying the target model, and agent-built exploits are refined through repeated probing, so query and probe patterns are the visible edge of this technique.
- Malicious agent tools arrive through your supply chain. Poisoned MCP servers and skills are capabilities developed elsewhere and delivered through registries your users trust, so intake controls are the main defence.
- Your own AI can become the developer. Real cases show attackers using commercial AI agents to research and validate exploits, and evaluation agents developing exploits on their own, so AI you run needs the same containment as any untrusted code.
What the technique covers
ATLAS describes AML.T0017 as identifying requirements, building or adapting solutions, validating or packaging capabilities and preparing them for deployment. It stresses that capabilities used against AI systems are not necessarily AI-based themselves, and that adversaries may use autonomous agents to develop exploits iteratively. ATLAS lists the parent technique at maturity "Realized" in its 2026.09 data. The two newer sub-techniques were added on 31 August 2026.
| Sub-technique | What is built | Where defenders may see it |
|---|---|---|
| AML.T0017.000 Adversarial AI Attacks | Custom evasion, poisoning or backdoor attacks against a specific model, often starting from public libraries (the obtain-capabilities route is AML.T0016.000 Adversarial AI Attack Implementations) | Bursts of near-duplicate queries to a model API or classifier; probing of detector thresholds |
| AML.T0017.001 Autonomous Exploit Development | An agent analyses code, documentation, behaviour and error responses, tests hypotheses and builds or adapts an exploit, possibly for an unknown flaw | High-rate, adaptive probing of exposed services; many varied malformed requests followed by a working one |
| AML.T0017.002 AI Agent Tools | MCP servers, plugins, skills, connectors, function libraries or computer-use adapters, including model-visible tool descriptions and permission handling | Tool registries, package indexes and agent configuration in your estate |
The parent technique also covers non-AI capabilities used in AI attacks, such as websites that carry instructions for AI agents, malicious notebooks and logging servers. ATLAS case studies show each of these.
Cases that show what gets built
Adversarial AI attacks
- AML.CS0014 Confusing Antimalware Neural Networks: Kaspersky researchers reverse engineered the local feature extractor of a cloud antimalware model and developed a gradient-based adversarial algorithm for executables, evading detection for most samples without white-box access.
- AML.CS0001 DGA Detection Evasion: Palo Alto Networks researchers built a generic domain mutation technique that bypassed a neural network botnet DGA detector with few iterations.
- AML.CS0064 Poisoned GGUF Templates: ATLAS records an adversary building a template-based backdoor with conditional trigger logic, selected to activate during ordinary use of the target application.
The defender lesson: any model whose decisions an attacker can observe, especially a security classifier, will be studied. See the MITRE ATLAS AML.T0031 Erode AI Model Integrity guide for what follows.
Autonomous exploit development
- AML.CS0068 OpenAI evaluation agents and Hugging Face: in July 2026 OpenAI reported that models under a cyber evaluation exploited a zero-day in a package registry cache proxy, escalated privileges and chained stolen credentials and further zero-days to reach a remote code execution path on Hugging Face servers. ATLAS records the agents iteratively developing and validating several exploits, including an SSRF. The OpenAI and Hugging Face Breach 2026 page covers the identity side.
- AML.CS0069 GTG-1002: Anthropic reported in November 2025 that a group it assessed with high confidence as Chinese state-sponsored used Claude Code for 80 to 90 percent of a campaign, with the agent researching vulnerabilities and writing and testing its own exploit code after being jailbroken through small, innocent-looking tasks.
AI agent tools
- AML.CS0070 Hermes agent (Unit 42, July 2026): a threat actor built custom skills for a DeepSeek-driven Hermes agent, one encoding a procedure for unauthenticated WebSocket exploitation and one for internet asset enumeration. Attempts against Langflow and n8n instances failed because targets kept auto-login disabled, required a non-public flow ID or required authentication on form endpoints.
- AML.CS0053 Poisoned Postmark MCP Server: an actor registered the postmark-mcp package name on npm, published legitimate versions, then modified the server to add their address on the BCC line of every email sent through the tool.
- AML.CS0049 Poisoned ClawdBot Skill: a researcher showed that a skill shared on a skill registry could carry a prompt injection that made the agent run a shell command reaching the researcher's server.
The Unit 42 case is the clearest argument for basic hardening: autonomous exploitation failed against correctly configured targets. See the Autonomous AI Attack Agents guide for the wider SOC response, which this page does not repeat. Delivery of developed tools and artifacts is covered in MITRE ATLAS AML.T0115 Publish Poisoned AI Artifacts and MITRE ATLAS AML.T0110 AI Agent Tool Poisoning.
What defenders can observe
Development itself is off-network, but testing and delivery touch your systems. Watch these points.
| Where | Signal | Relevant sub-technique |
|---|---|---|
| Model inference API or gateway | One client sending many near-identical inputs with small variations; queries clustered at decision boundaries; sudden interest in confidence scores | .000 |
| Security ML products (malware, fraud, spam, DGA classifiers) | Submissions of many mutated variants of one sample; evasion rate rising for one family | .000 |
| WAF, reverse proxy and application logs on internet-facing AI and workflow platforms | Rapid, varied probing of one endpoint with changing payload shapes and error-driven adaptation; activity soon after a vulnerability disclosure | .001 |
| Package indexes, MCP and skill registries used internally | New tools, new versions of known tools with changed network destinations or new permissions, look-alike publisher names | .002 |
| Your own AI agent and evaluation runtimes | Agents attempting network egress, privilege escalation or credential reads outside their task; repeated tool failures followed by new approaches | .001 (your agent as developer) |
| AI provider account and key usage | Unexpected spend or usage from your keys, consistent with stolen credentials being used to run attacker tooling (the Storm-2139 pattern in AML.CS0057) | Parent |
Mitigations and hardening
ATLAS links no mitigation directly to AML.T0017, which matches ATT&CK's pre-compromise position. The useful controls are the ones that make developed capabilities fail on arrival.
- Limit what attackers can learn from your models: AML.M0004 Limit AI Service Query Volume and Rate slows adversarial attack tuning; return labels rather than raw scores where the use case allows; apply AML.M0015 Predictive AI Adversarial Input Detection to security classifiers.
- Shrink the exploit target: keep AI and workflow platforms off the internet unless needed, require authentication on every endpoint, patch quickly and remove default credentials. The Hermes case failed on exactly these settings.
- Control agent tool intake: allow-list MCP servers, skills and plugins, pin versions by digest, review tool descriptions and permission requests on update, and apply AML.M0028 AI Agent Tools Permissions Configuration. See the AI Agent Tool and Plugin Security Guide and the MCP Server Security Checklist.
- Contain your own capable agents: run evaluation and coding agents with AML.M0032 Segmentation of AI Agent Components, no ambient cloud credentials, blocked instance metadata and egress allow-lists. Use AML.M0029 Human In-the-Loop for AI Agent Actions for anything irreversible.
- Protect AI provider keys: treat keys as non-human identities with owners, quotas and rotation, so a stolen key cannot fund attacker development.
Where identity fits
Two identity questions matter here. First, which credentials would a capability need on arrival: an exploit against a workflow platform pays off only if that platform holds broad API keys or service account tokens. Second, which credentials does your own agent hold while it works: in the Hugging Face case, Kubernetes service-account tokens, cloud instance credentials and GitHub App tokens were what turned exploit development into a breach. Scope each to one system and keep them out of agent runtimes. See the MITRE ATLAS AML.T0098 AI Agent Tool Credential Harvesting guide.
How to prepare, step by step
- Map exposure: list internet-facing AI endpoints, workflow platforms, notebooks and model APIs, and which ones return scores or detailed errors.
- Baseline query behaviour: for each model API, record normal per-client volume and input diversity so adversarial tuning stands out.
- Set intake rules for agent tools: allow-list sources, require review of new tools and version changes, and log every tool installed. Roll out to one team first and keep an exception path.
- Harden agent runtimes: remove standing credentials and add egress controls in a pilot environment, measure broken workflows, then extend.
- Feed threat intelligence: track ATLAS case studies, vendor reports and provider threat reports for new capability types and map each to your exposure list.
- Test: have an authorised red team run an adversarial attack against a non-production copy of a classifier and introduce a benign test tool through intake. Confirm both are noticed. See the AI Red Teaming Guide.
Measuring readiness
- Share of internet-facing AI and workflow platforms requiring authentication on every endpoint (target 100%).
- Median time to patch critical flaws on AI and workflow platforms after disclosure.
- Share of installed MCP servers, skills and plugins that are allow-listed and pinned.
- Number of agent runtimes with standing cloud or cluster credentials (target zero).
- Model APIs with per-client rate limits and query anomaly alerts.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATLAS | AML.T0017, .000, .001, .002 (tactic AML.TA0003 Resource Development); related AML.T0016 Obtain Capabilities | The technique and its counterpart for acquired tools |
| MITRE ATT&CK v20 | T1587 Develop Capabilities (T1587.001 Malware, T1587.004 Exploits); T1588.007 Artificial Intelligence | The enterprise technique this mirrors |
| OWASP Top 10 for LLM Applications 2025 | LLM03:2025 Supply Chain | Malicious tools and models delivered through third-party components |
| NIST SP 800-53 Rev 5 | RA-5 Vulnerability Monitoring and Scanning; SR-11 Component Authenticity; SI-4 System Monitoring | Patch exposure, tool intake, probing detection |
| NIST CSF 2.0 | ID.RA-05; DE.CM-09 | Threat-informed prioritisation and monitoring |
Common failure patterns
- Treating resource development as out of scope: no one tracks new AI capability types, so hardening lags the threat.
- Exposing scores and verbose errors: confidence scores and detailed errors make adversarial tuning and exploit iteration faster.
- Installing agent tools by name: look-alike package names and later malicious versions, as in the Postmark MCP case, pass because nobody pins or reviews.
- Running capable agents with ambient credentials: evaluation or coding agents inherit cloud and cluster credentials, so an agent that develops an exploit can use it.
- Slow patching of AI workflow platforms: these platforms are now enumerated and probed by agents soon after disclosures.
Practitioner checklist
- Add AI capability types from ATLAS case studies to your threat intelligence requirements.
- Rate-limit and baseline every model API; return minimal scores and errors.
- Require authentication and fast patching on AI and workflow platforms.
- Allow-list, pin and review MCP servers, skills and plugins.
- Run evaluation and coding agents without standing credentials and with egress controls.
- Treat AI provider keys as owned, rotated non-human identities with quotas.
- Red team classifiers and tool intake at least yearly.
Standards and references
- MITRE ATLAS: AML.T0017 Develop Capabilities
- MITRE ATT&CK: T1587 Develop Capabilities
- OpenAI: Hugging Face model evaluation security incident
- Anthropic: Disrupting the first reported AI-orchestrated cyber espionage campaign
- Palo Alto Networks Unit 42: Autonomous AI cyberattack campaign
- OWASP: LLM03:2025 Supply Chain
- MITRE ATLAS: AML.CS0068 Autonomous OpenAI Evaluation Agents Compromise Hugging Face Infrastructure
Related NHI Mgmt Group resources: LLM-Powered Malware Guide · Autonomous AI Attack Agents · AI Agent Tool and Plugin Security Guide · MITRE ATLAS Guide