How should teams choose F1 score instead of accuracy for imbalanced classification problems?

Why F1 is the better metric when class balance is skewed

Accuracy is a useful summary only when the classes are fairly balanced and the cost of both error types is similar. In an imbalanced problem, a model can score well by predicting the majority class most of the time, while still failing the cases you care about. F1 is better when the minority class matters because it evaluates precision and recall together.

The practical question is whether a “mostly correct” model is actually useful. If the positive class is rare, accuracy can be inflated by true negatives that do little to help the decision you are making. F1 shifts attention to whether the model finds positives reliably and whether those positive predictions are trustworthy.

What F1 tells you that accuracy hides

F1 is the harmonic mean of precision and recall, so it penalises a model that is strong on one and weak on the other. That matters when the minority class is operationally important, because you often need both enough detection and acceptable precision. A model with high accuracy but poor recall may look good in aggregate and still miss too many relevant cases.

Accuracy also gives true negatives the same weight as positive-class outcomes, which can mask poor minority-class performance. F1 deliberately ignores true negatives, so it is more sensitive to how well the model handles the class of interest. That makes it a better fit for screening, detection, triage, and other problems where the positive class drives the business decision.

F1 is not a universal replacement for accuracy. If false positives and false negatives have very different costs, or if the negative class itself matters as much as the positive class, you may need additional metrics. In practice, F1 is most useful when the evaluation goal is “how well do we identify the rare thing?” rather than “how often are we right overall?”

When accuracy is still the right signal

Accuracy remains appropriate when classes are reasonably balanced and the errors have similar consequences. It is also useful when the task is broad correctness rather than minority-class detection, or when a simple headline metric is enough for a first pass. The mistake is not using accuracy at all, but treating it as sufficient when class skew makes it misleading.

Teams should also remember that a single metric rarely captures the full picture. F1 helps when imbalance is the central issue, but it does not show calibration, threshold behaviour, or the actual trade-off between missed positives and false alarms. If the decision threshold can be tuned, the threshold itself is often as important as the metric you report.

For teams that need a stronger framing around data handling and evaluation discipline, the NIST Privacy Framework is a useful reference point for structuring measurement around risk, data use, and outcomes rather than relying on a single headline score.

Practitioner Guidance

What to verify: Before choosing a metric, check the class prevalence and the business cost of each error type. If the positive class is rare and missing it is materially harmful, accuracy is usually the wrong primary measure.

Decision rule: Use accuracy for balanced problems with comparable error costs; use F1 when you need one number that reflects minority-class detection quality. If recall is critical but precision also matters, F1 is usually the better default than accuracy.

What practitioners underestimate: A high accuracy score can hide a model that is functionally useless for the real task. The more skewed the data, the more important it is to inspect class-specific results, not just the overall rate.

Practitioner takeaway: Choose the metric that matches the decision, not the one that looks best on a dashboard. For imbalanced classification, F1 is often the safer summary because it forces you to confront whether the model actually finds the rare cases well enough to matter.

Related resources from NHI Mgmt Group

• F1 Score
• Classification
• How should ML teams choose PR AUC for heavily imbalanced classification problems?
• Why does F1 score give a more honest signal than accuracy in skewed datasets?