Should organisations use ROC curves or precision-recall curves for imbalanced data?

Why ROC and precision-recall curves answer different questions

ROC and precision-recall curves both evaluate ranking quality, but they emphasise different parts of model behaviour. ROC plots true positive rate against false positive rate, so it is useful when you want to understand how well a model separates classes overall. Precision-recall focuses on the positive class, which becomes more informative when the class you care about is rare.

For imbalanced data, that distinction matters because a model can achieve a strong ROC result while still producing many false positives relative to the number of true positives. Precision-recall curves make that trade-off visible. They are often the more practical view when the downstream decision depends on finding rare events efficiently rather than simply ranking positives above negatives in the abstract.

A useful way to think about it is that ROC answers “how well does the model distinguish classes?” while precision-recall answers “how useful are the predicted positives?” Those are related but not identical questions, so the right curve depends on the operational decision you need to support.

When imbalanced data makes ROC look better than it is

Class imbalance changes the meaning of false positives and true positives. When negatives dominate, the false positive rate can remain low even if the absolute number of false positives is large, because the denominator is so large. That can make ROC curves look reassuring even when the model creates too much noise for a real workflow.

Precision-recall curves are more sensitive to that problem because precision directly reflects how many predicted positives are actually correct. If the positive class is rare, precision often exposes whether a model is generating a usable alert stream or just a long list of low-value hits. That is why they are usually the better diagnostic view for highly skewed datasets.

ROC still has value. It is often helpful for comparing models, understanding separation across thresholds, and spotting whether one model dominates another broadly. But for rare-event detection, the curve can hide the practical cost of false alarms, so it should not be the only metric you rely on.

Choose the curve that matches the business cost of errors

The better chart depends on what failure is expensive in your workflow. If false positives create review burden, manual escalation, or user friction, precision becomes central and precision-recall is usually the better lens. If missing positives is worse and you need to understand threshold behaviour across a wide range of operating points, ROC can still be a useful companion view.

In practice, many teams use both. ROC helps with broad model comparison, while precision-recall helps set a threshold that is defensible for the actual deployment context. That combination is especially useful when the model will be used as a screening layer rather than a final decision engine.

The important point is not to treat either curve as universally superior. The metric should reflect the real question: how well the model ranks cases, or how actionable the predicted positive set is at the threshold you intend to use.

Practitioner Guidance

What to verify: Check the base rate of the positive class before choosing your primary evaluation curve. If positives are rare, inspect precision at the recall levels that matter operationally, not just the overall area under the curve.

Decision rule: Use precision-recall as the primary evaluation when you care about the quality of positive predictions in a skewed dataset. Keep ROC as a secondary comparison tool when you still need to compare separability across models or thresholds.

What good looks like: A useful model produces a precision-recall shape that remains acceptable at the recall you can afford, with false positives low enough that the output is actually actionable for the team consuming it.

Practitioner takeaway: For imbalanced data, evaluate the metric that reflects the cost of being wrong in the real workflow, not the one that simply looks smoother or more familiar.

Related resources from NHI Mgmt Group

• How do teams decide between ROC and precision-recall curves?
• How do organisations reduce exposure for data in use?
• How should organisations use data observability for AI reliability and audit readiness?
• How should organisations use data profiling before AI deployment?