An adversary does not have to beat your AI system to hurt you: it can bury the people who review its output. MITRE ATLAS tracks this as AML.T0046 Spamming AI System with Chaff Data, in the Impact tactic (AML.TA0011). ATLAS describes two forms: spamming an AI system with chaff data that increases the number of detections, so analysts waste time reviewing and correcting wrong inferences, and spamming AI agents with excessive low-severity auditable events or actions that need a human in the loop, wasting reviewers' time. ATLAS rates the technique as feasible. This guide is for SOC leads, fraud and trust and safety teams, and owners of AI agents with approval gates: how the attack works, how to tell it apart from neighbouring techniques, how to design queues that survive it, and how to respond. For the framework, see the MITRE ATLAS Guide.
Key takeaways
- The target is human attention. AML.T0046 succeeds when reviewers drown, approve by reflex or switch detection off, not when the model breaks.
- Two surfaces, one pattern. Detection models raise false alarms on chaff inputs; agents raise floods of approval requests and audit events. Both turn a safeguard into a cost.
- ATLAS maps two mitigations. AML.M0004 Limit AI Service Query Volume and Rate and AML.M0019 Control Access to AI Models and Data in Production. Queue design and risk-tiered approval are our additions (NHI Mgmt Group).
- Chaff is often cover. A flood can hide the one malicious item or approval request that matters, so triage the flood for a needle, not only for volume.
- Measure the queue, not just the model. Alerts per analyst hour, approval decision time and rubber-stamp rates show the attack earlier than model metrics do.
How the technique works
AI systems that flag things for human review have a fixed downstream capacity: a fraud team, a moderation queue, a SOC shift, an approver on call. AML.T0046 aims at that capacity.
- Chaff against detection models: the adversary submits inputs that the model will flag but that are harmless, such as transactions, files, messages or network events crafted to look suspicious. Each one creates work. ATLAS notes that a replicated model built with AML.T0005.001 Train Proxy via Replication can be used to craft adversarial data for spamming with chaff, which lets the adversary tune inputs offline until they reliably trigger detections.
- Floods against AI agents: the adversary causes an agent to generate many low-severity auditable events, or many actions that require human approval. Inputs can arrive through any channel the agent reads, such as email, tickets, chat or documents. The reviewer faces a queue of plausible requests and either slows the business down or starts approving without reading.
The OWASP Agentic Security Initiative lists the same agent-side risk as T10 Overwhelming Human in the Loop in its Agentic AI Threats and Mitigations document (v1.0, February 2025). Approval fatigue as a design problem is covered in the Human-in-the-Loop for AI Agents Guide; this page treats it as an attack.
Telling it apart from neighbouring techniques
| Technique | What the adversary exhausts | Typical sign |
|---|---|---|
| AML.T0046 Spamming AI System with Chaff Data | Human review time and trust in alerts | Detections or approval requests rise while true positive rate falls |
| AML.T0029 Denial of AI Service | Compute and availability | Latency, errors and capacity alarms on the model service |
| AML.T0034 Cost Harvesting | Money spent on inference | Spend and token use rise without matching business activity |
| AML.T0015 Evade AI Model | Detection itself | Malicious items pass with no alert |
Campaigns often combine these. A chaff flood followed by an evasion attempt is the case to plan for: the flood lowers attention, then the real item goes through.
Where it shows up
| System | What chaff looks like | Who pays |
|---|---|---|
| Fraud and transaction scoring | Many small, odd but legitimate transactions or sign-ups that trip risk models | Fraud analysts; legitimate customers held in review |
| Email and content filtering | Benign messages crafted to score as suspicious, filling quarantine review | Messaging and trust and safety teams |
| ML-based intrusion and malware detection | Harmless files or traffic patterns that trigger detections | SOC analysts |
| AI-assisted SOC triage | Noise that the triage model escalates rather than closes | Tier 2 analysts |
| AI agents with approval gates | Inbound emails, tickets or documents that cause the agent to request approval for many routine actions | Approvers and business owners |
| Agent audit and monitoring pipelines | Floods of low-severity tool calls and events that bury the meaningful ones | Agent operations and the SOC |
Detection
The earliest signals are in queue and workflow data, joined to the identity of whoever caused each item. Our recommended signals (NHI Mgmt Group):
| Signal | What it suggests | Data source |
|---|---|---|
| Detections rise sharply while confirmed true positives stay flat | Chaff against a detection model | Alert and case outcome records |
| Many detections share a source, account, sender, template or feature profile | Coordinated chaff rather than organic noise | Model telemetry with input clustering |
| Approval requests per agent run or per inbound item jump | Agent being steered into approval floods | Agent orchestration and approval logs |
| Approval decision time drops and approval rate approaches 100 per cent | Reviewer fatigue, rubber stamping | Approval system audit trail |
| A high-impact item arrives during or just after a flood | Flood used as cover | Timeline correlation across queues |
| Flood sources are new, unauthenticated or low-reputation identities | Disposable access used for spamming | Account, sender and API key records |
ATLAS mitigation AML.M0024 AI Telemetry Logging supports all of these: for agents it calls for logging intermediate steps, data access, tool use and agent identity, which is what lets you trace a flood back to its trigger. In D3FEND terms the analytics sit under D3-UBA User Behavior Analysis.
Metrics
- Alerts per analyst hour and its week-on-week change per AI detector.
- True positive rate per detector, tracked daily, so a fall stands out.
- Approval requests per agent per day and median approval decision time.
- Rubber-stamp rate: share of approvals made in under a set number of seconds (set the threshold yourself from your baseline).
- Flood containment time: from the start of an abnormal surge to rate limiting or blocking its source.
Prevention and hardening
| Control | How it applies | Trade-off |
|---|---|---|
| AML.M0004 Limit AI Service Query Volume and Rate (ATLAS-mapped) | Limit how many inputs an account, sender, key or tenant can submit in a period, and how many approval requests one source can trigger | Bulk legitimate sources need explicit higher limits |
| AML.M0019 Control Access to AI Models and Data in Production (ATLAS-mapped) | ATLAS notes authentication on production models helps prevent anonymous chaff spam; require authenticated, attributable sources for anything that can create review work | Some inputs, such as inbound email, cannot be authenticated; treat them as lowest trust |
| AML.M0029 Human In-the-Loop for AI Agent Actions (our addition) | ATLAS advises that approval should follow the degree of consequence of the task; tier approvals so routine actions do not reach a human at all | Fewer approvals means more reliance on policy and logging |
| Queue design (our addition) | Group similar alerts or requests into one case, rank by risk, and cap how much one source can contribute to a queue | Grouping can hide a distinct item if the grouping key is too broad |
| Detector tuning with outcome feedback (our addition) | Feed analyst verdicts back to the model or rules so chaff patterns are recognised and down-ranked | Feedback loops can be poisoned; review what is learned |
Designing review queues that withstand flooding
- Per-source budgets: each account, sender or agent gets a share of the review queue. Excess items are held, sampled or auto-closed with logging, never silently dropped.
- Batch approvals carefully: letting an approver accept many requests at once ends the flood and also ends real review. Allow batching only for low-impact action types, and never for payments, permission changes or data exports.
- Escalate the anomaly, not each item: when a surge is detected, raise one incident for the surge and route its items to a separate queue, so the normal queue keeps working.
- Protect high-impact paths: keep a separate, small queue with a second approver for the actions an attacker would hide in the noise.
Decision guidance
- Small teams: start with per-source limits and grouping. Both are cheap and limit the damage of any flood.
- Large SOCs and fraud teams: add detector-level true positive tracking and surge detection, and plan surge staffing for incidents.
- Agent owners: reduce what needs approval before scaling agents. An agent that asks for approval on every step is easy to flood and trains approvers to click through. See the Alert Fatigue Reduction Guide for tuning practice that applies to both alerts and approvals.
Where identity fits
Every flood has a source identity: a customer account, an email sender, an API key, or an agent acting with its own service identity. Our recommendations (NHI Mgmt Group):
- Attribute every review item: record which human, key or agent identity caused it, so limits and suspension can act on the source.
- Give each agent its own identity: a shared service account across agents hides which one is being steered into a flood. See the Agentic AI Identity Guide.
- Protect approver accounts: approval floods are also a route to MFA-fatigue style mistakes. Bind approvals to strong authentication and show the approver who and what triggered the request.
Response
- Declare a surge: when metrics cross thresholds, open one incident for the flood rather than working items one by one.
- Find the needle: before bulk-closing anything, search the flood window for high-impact items, approvals for sensitive actions and items from sources outside the flood cluster.
- Contain the source: rate limit or block the accounts, senders, keys or input channels responsible, and pause agent actions that were triggering approvals.
- Clear the queue safely: bulk-close clustered chaff with a recorded rationale and a sample checked by a second analyst.
- Review approvals made during the flood: treat approvals made at abnormal speed as unverified and re-check the high-impact ones.
- Tune and record: adjust detectors and approval tiers, add the chaff fingerprint to detection, and log the incident against AML.T0046. See the AI Incident Reporting Guide.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATLAS | AML.T0046 in AML.TA0011; related AML.T0029, AML.T0034, AML.T0015; enabler AML.T0005.001 | Technique and neighbours |
| MITRE ATLAS mitigations | AML.M0004, AML.M0019 (mapped); AML.M0024, AML.M0029 (our additions) | Limits, access, logging, approval tiers |
| OWASP Agentic AI Threats and Mitigations | T10 Overwhelming Human in the Loop | Agent-side form of the attack |
| OWASP Top 10 for LLM Applications 2025 | LLM10 Unbounded Consumption | Volume controls for LLM-backed services |
| NIST SP 800-53 Rev. 5 | SC-5, SI-4, AU-6, IR-4 | Resource limits, monitoring, review, incident handling |
| NIST CSF 2.0 | DE.AE-02, PR.IR-04, RS.MI-01 | Event analysis, capacity, containment |
Common failure patterns
- Turning the detector off: a noisy model is disabled during the flood, which is often the adversary's actual goal.
- Approval for everything: agents gated on every step create the queue an adversary needs and teach approvers to click through.
- Bulk close without a search: the flood is cleared in one action and the malicious item inside it goes with it.
- No source attribution: alerts and approval requests cannot be traced to the account or agent that caused them, so nothing can be rate limited.
- Measuring only model accuracy: offline accuracy looks fine while the human queue collapses.
Practitioner checklist
- List every AI system that creates human review work, with its queue owner and normal daily volume.
- Record the source identity for every alert, case and approval request.
- Apply per-source limits to inputs and to approval requests, with documented higher tiers.
- Tier agent approvals by consequence so routine actions do not reach a human.
- Track true positive rate per detector and approval decision time per agent, with surge thresholds.
- Keep a separate protected queue for high-impact actions with a second approver.
- Write a surge runbook: declare, search for the needle, contain, clear with sampling, re-check fast approvals.
- Include chaff and approval-flood scenarios in AI red team exercises.
Standards and references
- MITRE ATLAS: Spamming AI System with Chaff Data
- MITRE ATLAS: Limit AI Service Query Volume and Rate
- MITRE ATLAS: Human In-the-Loop for AI Agent Actions
- OWASP: Agentic AI Threats and Mitigations
- OWASP: LLM10:2025 Unbounded Consumption
- NIST: AI 100-2 E2025 Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
- NCSC: Guidelines for secure AI system development
Related NHI Mgmt Group resources: Human-in-the-Loop for AI Agents Guide · Alert Fatigue Reduction Guide · OWASP LLM Top 10 LLM10 Unbounded Consumption Guide · MITRE ATLAS Mitigations Guide