Why can Shapley values produce misleading explanations if the baseline choice is weak?

Why the baseline is doing more work than the scorecard suggests

Shapley values are only as good as the “missing feature” world they assume. If the baseline is weak, the method can assign credit against a counterfactual that the model never really sees in production, so the numbers may be mathematically consistent but operationally misleading. The result is often a clean explanation of an artificial scenario, not a faithful explanation of the real decision path.

That is why baseline choice matters as much as the attribution method itself. A baseline that is out of distribution, overly generic, or detached from how the model is actually used can make features look more or less important than they are. In practice, the explanation may describe how the model behaves relative to an imagined absence, while the practitioner wants to know what actually drove the prediction in context.

A weak baseline also changes the interpretation of “importance” itself. Shapley values compare the prediction to a reference world, so if that reference world is unrealistic, the marginal contributions reflect the gap between two different regimes rather than the model’s genuine reasoning. CIS Benchmarks are a useful analogy here: a hardening baseline only helps when it reflects the environment you actually operate, not a generic ideal that never existed in the first place.

Risk and Threat Considerations

Weak baselines can create false confidence because the explanation appears precise even when the reference world is unrealistic. That matters when attribution output is used for model validation, incident review, or business decisions, because the apparent clarity can hide whether the model is responding to a real signal or to baseline artefacts.

Failure mechanism: If the baseline distribution is poorly chosen, the “feature absent” condition becomes a synthetic counterfactual that distorts marginal contributions. The explanation then overweights or underweights variables based on baseline mismatch, not actual predictive logic.

Impact: Teams may approve models, debug the wrong features, or communicate unsupported rationale to stakeholders. In regulated or high-stakes settings, that can undermine traceability and make later validation harder even when the model itself has not changed.

Practitioner Guidance

What to verify: Treat the baseline as a modelling decision, not a default setting. Verify that it reflects the operating population, feature dependencies, and decision context you actually care about, especially when the data are highly clustered or non-stationary.

Decision rule: If the baseline cannot be defended as representative of the real inference environment, use the explanation as directional only and avoid making causal claims from the Shapley ranking. When the baseline is sensitive, compare results across several plausible references and look for stable patterns rather than a single “best” attribution.

Practitioner takeaway: A Shapley explanation is only trustworthy when the counterfactual world is credible; otherwise, the output measures sensitivity to a made-up reference point more than it measures model logic.

Related resources from NHI Mgmt Group

• Why do validation metrics become misleading when data integrity is weak?
• Why can SHAP explanations become misleading in real datasets?
• How should teams decide whether Shapley values are suitable for a model explanation workflow?
• Why do Shapley explanations become unreliable in some tabular models?