Adversarial accuracy measures how many correctly classified samples stay correct after intentional perturbation. Empirical robustness measures the minimum perturbation needed to force a misclassification. The first is a pass rate under attack, while the second is a resistance threshold. Used together, they show whether a model merely survives tests or genuinely resists adversarial manipulation.
What each metric tells you about model behavior
Adversarial accuracy and empirical robustness answer related but different testing questions. Adversarial accuracy tells you the model’s pass rate under a specified attack or perturbation budget, so it is useful for comparing how often predictions survive a chosen test. Empirical robustness asks how far the input must be perturbed before the model fails, so it is a threshold measure of resistance rather than a pass/fail score.
That distinction matters because a high adversarial accuracy can hide shallow resilience if the attack budget is weak, while a strong empirical robustness result can show a larger safety margin even when a model still fails some perturbation tests. Used together, they tell you whether the model is merely holding up under one test condition or actually pushing the decision boundary away from likely attack behavior.
Why the difference matters in evaluation design
These metrics are not interchangeable because they depend on different evaluation choices. Adversarial accuracy is tied to the attack method, perturbation budget, and dataset slice you test, so the number is only as meaningful as the test protocol behind it. Empirical robustness is tied to the smallest successful perturbation you can find, which makes it more about search quality and attack coverage than a simple classification score.
For practitioners, the practical question is whether the test suite is broad enough to represent the real failure mode you care about. A model can look better on adversarial accuracy if the attacker is constrained, but still be brittle once the perturbation search is improved. Conversely, a model can show resistance in empirical robustness terms while still being vulnerable to a different attack family that the test never explored.
How to read both numbers in a model test report
Read adversarial accuracy as a survival ratio and empirical robustness as a resistance threshold. If the report gives only adversarial accuracy, you know how many examples stayed correct under that test, but not how close the model was to failure. If it gives only empirical robustness, you know how much perturbation was needed to break the model, but not how many examples were already vulnerable at a fixed budget.
The strongest interpretation usually comes from pairing them with clean accuracy and with the exact perturbation setup used in the test. That lets you separate ordinary classification quality from attack sensitivity. It also helps you compare models fairly, because two systems can have the same adversarial accuracy while one fails only after a much smaller perturbation than the other.
Risk and Threat Considerations
Weak adversarial testing can create a false sense of security, especially when a model is deployed in a setting where inputs can be manipulated, such as user content, OCR, ranking, or decision support. If the attack budget is too narrow or the test search is too shallow, adversarial accuracy may overstate resilience and hide practical evasion risk.
Failure mechanism: The evaluation uses a fixed perturbation budget or incomplete attack search, so it measures only the chosen test rather than the model’s true margin to misclassification.
Impact: Teams may deploy a model that appears robust in reports but remains easy to mislead under a slightly different perturbation strategy, which increases operational and abuse risk.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATLAS addresses the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATLAS | Adversarial AI Threat Matrix | Models adversarial AI techniques that shape robustness testing and attack assumptions. |
| Recommendation — Map evaluation findings to ATLAS techniques and test against representative AI attack paths. | ||
| NIST AI RMF | Generative AI Risk Management | Supports evaluating model resilience and attack assumptions within AI risk governance. |
| Recommendation — Document attack assumptions and measurement limits in your AI risk management process. | ||
| NIST CSF 2.0 | ID.RA-01 — Asset Vulnerabilities Are Identified and Recorded | Model robustness testing is a vulnerability identification activity for deployed AI systems. |
| Recommendation — Record model weaknesses exposed by adversarial testing and track them to remediation. | ||
Practitioner Guidance
What to verify: Check the perturbation budget, attack family, and search method before trusting either metric. If the evaluation does not state those clearly, the numbers are hard to compare and easy to overread.
What good looks like: A useful test report shows clean accuracy, adversarial accuracy at one or more realistic budgets, and empirical robustness across a range of attacks so you can see both pass rate and failure threshold. For model threat modeling, the MITRE ATLAS adversarial AI threat matrix is a good companion for connecting test results to concrete attack techniques.
Practitioner takeaway: Treat adversarial accuracy as a scenario-specific survival score and empirical robustness as a margin estimate, then judge the model by whether both remain defensible under realistic attack assumptions.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on November 5, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org