Macro recall is an aggregate recall measure across categories or change types, giving equal weight to each group rather than letting large groups dominate the result. It is useful when evaluating whether an agent performs consistently across mixed security scenarios, especially when some categories are much easier than others.
What Macro Recall Measures
Macro recall is useful when you want to know how well a system covers each category, not just how well it performs on the largest or most frequent one. It treats each group equally, so the score reflects whether weaker classes are being missed rather than being masked by dominant categories.
This makes the metric especially relevant in mixed security evaluations, where some scenario types may be rare but operationally important. A strong macro recall score suggests the agent is not only succeeding overall, but is maintaining coverage across the full set of categories being tested.
Why Equal Weighting Matters
Macro recall is a fairness-oriented aggregate. If one category has many more examples than another, a micro-averaged metric can look healthy even when the model struggles badly on the smaller group. Macro recall removes that imbalance by averaging recall per category, so each category contributes the same amount to the final result.
That design is valuable when category imbalance is expected. In security testing, for example, the hardest or least common change types can be the ones that matter most operationally, so a metric that hides failures in those groups can give a misleading sense of robustness.
How To Read The Score
A macro recall score is best interpreted as a consistency signal. Higher values indicate that recall is more even across the evaluated groups, while lower values usually mean one or more categories are being under-detected. The score alone does not show which group is failing, so it works best alongside per-category recall.
Because each group is weighted equally, the metric is sensitive to small categories. That is a feature, not a flaw, when the goal is to prevent an easy majority class from dominating the evaluation. It is less suitable when you only care about aggregate volume-weighted performance.
Where Macro Recall Is Most Useful
Macro recall is most useful in benchmark design, model comparisons, and regression testing for systems that must behave reliably across diverse scenario types. It helps answer a practical question: does the agent generalise across categories, or is it merely strong in the most common ones?
For security-oriented agent evaluation, this matters because uneven performance can hide blind spots in rare but important scenarios. Macro recall helps expose those blind spots earlier, especially when paired with a breakdown of each category’s recall so teams can see where the model is falling short.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack surface, NIST CSF 2.0 and NIST AI RMF set the technical controls, and ISO/IEC 42001:2023 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.OV-01 — Oversight of Cybersecurity Risk | Macro recall supports oversight of evaluation quality across security scenarios. |
| ID.RA-01 — Asset Vulnerabilities Are Identified and Documented | Macro recall helps surface weak coverage in specific test categories that may signal blind spots. | |
| Recommendation — Track evaluation metrics that reveal uneven security performance across scenario groups. Use per-category testing to identify where detection or control coverage is uneven. | ||
| OWASP Agentic AI Top 10 | ASI08 — Cascading Failures | Macro recall is relevant to judging whether agent performance degrades in harder scenario categories. |
| Recommendation — Measure agent performance across scenario classes to find failure concentration in edge cases. | ||
| NIST AI RMF | MAP — Map | Macro recall fits AI risk mapping by comparing performance across distinct use-case groups. |
| Recommendation — Map evaluation coverage across scenario groups so weaker classes are not hidden by averages. | ||
| ISO/IEC 42001:2023 | 8.2 — AI risk treatment | Macro recall supports AI risk treatment by highlighting uneven performance across tested categories. |
| Recommendation — Treat category-level evaluation gaps as inputs to AI risk treatment decisions. | ||
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on October 31, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org