Baseline Sensitivity

What Baseline Sensitivity Means in Practice

Baseline sensitivity describes how much an explanation changes when the reference point, comparison set, or counterfactual input changes. It is a practical property of explanation stability, not a statement about model accuracy or prediction quality.

For governance, the key question is whether the explanation remains materially similar under reasonable baseline choices. If it does not, the explanation may be too dependent on an arbitrary starting point to support audits, reviews, or user-facing decisions.

Why Baseline Choice Matters

Baseline selection can change what an explanation appears to highlight. In attribution-style methods, for example, different references can shift importance across features even when the underlying model output is unchanged. That means two valid runs can produce different stories about the same decision.

This is why baseline sensitivity is often treated as a robustness concern. A low-sensitivity explanation is easier to compare across reviews, while a high-sensitivity explanation can be misleading if readers assume the explanation is more stable than it really is.

Teams often discover that the issue is not the explanation method alone, but the reference policy behind it. A baseline should be chosen with the same care as the explanation technique itself, because the reference point is part of the result.

Where Baseline Sensitivity Breaks Down

Baseline sensitivity becomes a problem when explanations are used as evidence of fairness, compliance, or operational correctness. If the same case produces materially different explanations under different counterfactuals, reviewers may overstate confidence in the explanation and understate uncertainty.

It also creates comparability problems. A high-sensitivity explanation may be valid for one input but not useful as a general interpretation method, because the narrative depends too heavily on a particular reference choice. The CIS Benchmarks are a useful reminder that stable security decisions usually depend on clearly defined baselines, not ad hoc ones.

In operational settings, unstable explanations can frustrate incident review, model validation, and user trust. If the explanation changes materially with small baseline changes, the organisation should treat it as a signal to examine explanation design, not as a minor presentation issue.

How to Interpret It in Governance and Review

Baseline sensitivity should be read as a reliability check on the explanation layer. It does not automatically invalidate the model, but it does indicate that interpretation may depend on choices that are invisible to most readers.

Good governance practice is to document which reference points are used, why they were chosen, and how sensitive the explanation is to reasonable alternatives. That makes the explanation easier to compare, challenge, and reproduce during review. For organisations that already rely on control-oriented baselines, the idea is consistent with how the NIST Cybersecurity Framework 2.0 treats repeatable, governed processes as part of trustworthy operations.

When explanation stability matters more than novelty, teams should prefer simpler or better-normalised explanation setups over more impressive but fragile ones. The practical goal is not to eliminate all sensitivity, but to keep it within a range where the explanation still supports decision-making.

Related resources from NHI Mgmt Group

• What breaks when DLP depends only on sensitivity labels?
• Why does leaving Linux outside the passwordless baseline increase identity risk?
• When should organisations expand beyond the baseline controls in NIST 800-53?
• What breaks when access findings are not paired with data sensitivity?