Use F1 when the positive class is rare, when false negatives matter, or when you need a clearer view of minority-class performance. Accuracy can look strong even when a model misses the cases that matter most. F1 ignores true negatives and forces you to evaluate the balance between precision and recall, which is usually the right lens for skewed data.
Why F1 Is the Better Lens for Skewed Classification
Accuracy answers a different question than most imbalanced problems actually pose: it tells you how often the model is right overall, not whether it finds the rare cases that matter. When the positive class is sparse, a model can achieve impressive accuracy by predicting the majority class almost all the time, while still failing the operational goal. F1 is useful because it forces teams to balance precision and recall instead of hiding minority-class misses behind true negatives. That makes it a better fit when the cost of missing positives is material or when the dataset is heavily skewed. For teams comparing model performance across skewed labels, F1 also makes evaluation less sensitive to class prevalence than raw accuracy does. The practical issue is not whether the model looks good on paper, but whether it performs credibly on the class that drives the decision. In practice, many teams discover this only after a model with “high accuracy” fails on the cases they were most hoping to catch.
A useful rule of thumb is that F1 becomes more informative when a good result requires the model to both find positives and avoid flooding reviewers with false alarms. If you are scoring service tickets, fraud flags, rare defects, or abuse patterns, accuracy often overstates value. This is why practitioners usually treat accuracy as a sanity check, not the decision metric, once skew becomes meaningful.
How F1 Changes the Evaluation Workflow
F1 is the harmonic mean of precision and recall, so it penalises a model that performs well on one and poorly on the other. That matters in imbalanced settings because the majority class can dominate accuracy even when minority-class recall is weak. If the team cares about catching positives without generating too many false positives, F1 gives a clearer single-number summary of that trade-off. The metric is most useful after the team has already decided that true negatives are not the main source of business value.
In practice, teams should compare F1 alongside precision and recall, not instead of them. F1 hides the direction of the trade-off, which means two models can share the same score while one is much more conservative and the other much more aggressive. That distinction matters when the downstream process has limited review capacity or when false negatives are much more costly than false positives. For highly skewed data, macro or weighted F1 may also be more meaningful than plain accuracy because they surface minority-class behaviour more directly.
- Use F1 when the positive class is operationally important and the negative class is plentiful.
- Use precision when false alarms are expensive, and recall when misses are expensive.
- Use F1 as a summary metric, then inspect the confusion matrix before trusting the result.
- Use class-specific or macro-averaged F1 when the minority class is what the team actually needs to protect.
For teams that need a formal control lens around evaluation and reporting, NIST’s control families on monitoring, analysis, and assessment provide a useful governance analogue, and the NIST SP 800-53 Rev 5 Security and Privacy Controls is a relevant reference for disciplined measurement and review. NHIMG’s Ultimate Guide to NHIs is also useful when the “positive class” is a risky identity population that is easy to undercount. These controls tend to break down when teams optimise a single metric without checking whether the evaluation threshold matches the actual operational cost of misses versus false alarms.
Common Variations and Edge Cases
Tighter evaluation criteria often increase complexity, requiring teams to balance interpretability against decision quality. F1 is not always the best choice just because the data are imbalanced. If true negatives matter a lot, or if the team needs probability calibration rather than a binary classification summary, accuracy, ROC-AUC, PR-AUC, or cost-based metrics may be more appropriate. There is no universal standard for this yet; the right metric depends on the downstream action and the class imbalance pattern.
One important edge case is extreme rarity with very low prevalence. In those settings, even F1 can be misleading if the team does not set an operating threshold carefully, because the score can improve while real-world utility remains weak. Another common issue is multi-class imbalance, where per-class F1 or macro-F1 usually matters more than a single micro-averaged number. If the business consequence of a false negative is severe, teams should avoid treating overall accuracy as evidence of readiness.
What practitioners underestimate: metric choice shapes behaviour. If the team reports accuracy to stakeholders who care about misses, they may get the wrong model approved. If they report F1 without explaining precision and recall, they may hide the exact failure mode the business needs to manage.
Risk and Threat Considerations
imbalanced classification creates decision risk when the majority class masks failures in the minority class. The main exposure is false confidence: a model can look strong in aggregate while systematically missing the cases that carry the highest operational, compliance, or fraud impact.
Failure mechanism: Accuracy rewards correct negatives at scale, so a skewed dataset can produce a high score even when the model rarely identifies positives. That becomes dangerous when decision thresholds, reviewer capacity, or alerting logic are tuned to the wrong metric and the minority class is effectively invisible.
Impact: Teams can under-detect rare but important events, route too little attention to the cases that matter, and approve models that are not fit for purpose in production.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK address the attack and risk surface, while NIST CSF 2.0 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | ID.IM-01 — Organizational Context | Class imbalance changes what 'good' means for a model's business context. |
| DE.CM-08 — Detection Processes | Evaluation metrics should reveal whether rare events are being detected effectively. | |
| Recommendation — Define success metrics that reflect the risk of missing minority-class events. Measure minority-class detection quality instead of relying on overall accuracy. | ||
| CIS Controls v8 | 8.1 — Establish and Maintain Audit Log Management Process | Reliable review of model outcomes depends on auditable evaluation and review records. |
| Recommendation — Retain classification outcomes and threshold decisions for review and tuning. | ||
| MITRE ATT&CK | T1071 — Application Layer Protocol | Classification metrics matter when models screen for low-volume abuse signals among normal traffic. |
| Recommendation — Tune detections to preserve minority signal that may indicate abuse or intrusion. | ||
Practitioner Guidance
What to prioritise: Start by mapping the real cost of a false negative versus a false positive. If misses are the bigger failure mode, F1 is usually a better default than accuracy, but only if the team still inspects precision and recall separately.
Decision rule: If a model can achieve high accuracy by predicting the majority class most of the time, do not use accuracy as the headline metric. Use F1 or class-specific F1 to judge whether the model is actually useful on the rare class.
What to verify: Check the confusion matrix at the chosen threshold, not just the summary score. A good f1 score is only meaningful if it is stable across validation sets and does not depend on a threshold that would be impractical in production.
Practitioner takeaway: Choose F1 when the question is “does the model find the minority cases well enough to act on?” and choose accuracy only when every class matters similarly and class balance is not distorting the result.
