Decision-Only Adversarial Attack

How Decision-Only Attacks Work

Decision-only attacks are built around the narrowest possible feedback channel: the model gives an output label, and nothing else. That makes them useful against systems that hide confidence scores, probabilities, or logits, because the attacker can still learn enough from repeated queries to infer where the decision boundary sits and how input changes affect the outcome.

In practice, the attack is less about a single clever prompt and more about iterative probing. Small perturbations are tested, outputs are observed, and the attacker gradually steers an input across the boundary until the model produces the desired misclassification or unsafe decision.

Why Decision-Only Feedback Is Still Dangerous

Security teams sometimes assume that withholding scores meaningfully reduces attack surface. It helps, but it does not remove the core problem, because a yes or no response still leaks structural information about the model’s behaviour. Once that information can be queried repeatedly, the model can be explored like any other system with an observable decision rule.

This is especially relevant for classifiers that gate access, moderation, fraud checks, spam filtering, malware detection, or other binary and multi-class decisions. If the output directly influences trust, access, or workflow routing, an attacker only needs to learn how to push the system over the threshold, not how to read its internal state.

For broader adversarial machine learning context, see the MITRE ATLAS adversarial AI threat matrix, which catalogs techniques used to probe, evade, and manipulate AI systems.

How Organisations Reduce Exposure

Exposure starts with the model’s interface. Rate limits, query quotas, anomaly detection, and challenge mechanisms can make boundary estimation much harder by increasing the cost of repeated probing. Input normalisation and output throttling can also reduce the precision of what the attacker learns from each query.

Defenders should also look at the surrounding system, not just the model. If the classifier sits behind an API, the access path, logging, and abuse controls matter as much as the model itself. For AI systems that expose tools, automation, or operational actions, the AI Agent Security Testing Checklist is a useful internal reference for evaluating repeated-query abuse, tool misuse, and regression testing around adversarial behaviour.

More broadly, the AI Threats and Model Security Guide covers the model-security controls that help reduce evasion, extraction, and adversarial probing risk across AI deployments.

Detection, Response, and Recovery After Abuse

Decision-only attacks are often visible only in the pattern of requests, not in a single malicious payload. That makes behavioural monitoring important: many similar queries, unusual input variation, repeated failures followed by success, and access concentrated around a single model endpoint are all signals that boundary probing may be underway.

If abuse is suspected, response should include reviewing logs for the query sequence, checking whether the model’s outputs were used to train follow-on attacks, and determining whether the same interface was used to test other inputs or model variants. Where the model influences downstream business or security decisions, post-incident review should also ask whether the attacker could have altered real decisions, not just demonstrated a proof of concept.

Authoritative threat intelligence and incident context can help shape that review. The CISA cyber threat advisories page is a useful starting point for tracking active threat patterns that may inform defensive prioritisation.

What This Means for Defenders of Decision Systems

Decision-only attacks show that output minimisation is not the same as security. A model can be resistant to score leakage and still remain vulnerable if an attacker can query it enough times to map its behaviour.

MITRE ATLAS Guide is the most direct navigation aid when you are mapping this kind of attack to known adversarial AI techniques and mitigations. The practical question is not whether the model reveals probabilities, but whether the decision path can be learned, steered, or abused through repeated interaction.

Why practitioners should care: The main defence is to treat repeated low-information queries as a security signal in their own right. If the system makes consequential decisions, the attack surface exists even when the model never exposes anything beyond the final label.

Related resources from NHI Mgmt Group

• Who should own the decision to restore data after an attack?
• What are the signs that an adversarial attack is affecting AI model outputs?
• What is the difference between attack path validation and control validation in adversarial testing?
• How should security teams define roles and decision authority before a ransomware attack?