Macro Recall

Expanded Definition

Macro recall is a fairness-sensitive evaluation lens for a mixed set of categories, change types, or task slices. Instead of letting high-volume classes dominate the score, it gives each group equal weight so the final result reflects consistency across the full distribution, not just strength on the largest bucket.

In security and AI assurance work, that matters when one scenario family is abundant and another is rare, but both are operationally important. Macro recall is not the same as overall recall, and it is not meant to replace per-category analysis. It answers a different question: whether performance holds up across the whole mix, including small or difficult categories that can otherwise be hidden by volume.

For readers who need a standards anchor, NIST SP 800-53 Rev. 5 is relevant as a governance reference for control-centric evaluation, but it does not define macro recall itself. The practical boundary is simple: macro recall is a measurement choice, not a control.

Examples and Use Cases

Macro recall is most useful when the evaluation set has uneven class sizes or uneven scenario difficulty. In practice, it helps teams see whether a model, detector, or review workflow is only strong on common cases.

• Evaluating an AI security classifier across phishing, credential theft, policy violations, and benign content, where benign examples may be numerous but not the point of the test.
• Comparing an agent's performance across several security workflows, such as ticket routing, alert triage, and change review, where one workflow may generate far more samples than the others.
• Measuring detection quality across attack families so a large family like credential abuse does not mask weak performance on lower-volume but high-impact cases.
• Reviewing content moderation or policy enforcement outputs where rare categories need equal attention to frequent ones.

A common tradeoff is that macro recall can look worse than overall recall when a system is highly tuned to frequent categories. That is not a flaw in the metric; it is a signal that the system may not be consistent enough for mixed-security use.

Security Implications

When macro recall is ignored, a system can appear strong while failing on the scenarios that matter most to assurance, safety, or containment. A model that performs well on abundant classes may still miss rare but consequential classes, producing a misleadingly optimistic picture of operational readiness.

The practical consequence is blind spots. In security workflows, that can mean missed malicious content, weak handling of uncommon attack variants, or uneven enforcement across policy categories. In agentic systems, the risk is especially visible when control logic is tested mostly on routine cases and underperforms on edge conditions that require restraint, escalation, or refusal.

A practitioner should watch for this symptom: a single aggregate score that hides category-level failure. If one slice is systematically weaker, macro recall makes that visible instead of averaging it away.

Domain and Governance Relevance

Macro recall matters in evaluation governance because it changes how success is defined. A team that reports only overall performance can end up optimizing for the largest segment, even when the business or security risk is concentrated in smaller segments. Macro recall forces a broader view of quality across categories that may not be equally represented but are equally relevant to trust.

In AI security and agent evaluation, that is especially important when the system is expected to behave consistently across varied prompts, threats, or policy conditions. For NHI and autonomous execution contexts, the same idea helps assess whether a control or model is reliable across different identity, permission, or action classes rather than only on the dominant workflow. That is useful when rare cases carry disproportionate blast radius.

For NHIMG readers, the governance question is not whether the metric is mathematically elegant. It is whether the metric reveals uneven assurance before an inconsistent system is allowed into security-sensitive operation.

Related resources from NHI Mgmt Group

• Why is syntax recall a weaker signal than architectural judgement?
• Why do precision and recall create blind spots in scanner selection?
• How do teams decide between ROC and precision-recall curves?
• Should organisations prioritise recall or precision when choosing a SAST platform?