Additive Feature Attribution

How additive feature attribution works

Additive feature attribution explains a model output by assigning each input feature a contribution value and then summing those contributions to reconstruct the prediction. That makes the explanation locally intuitive, because each feature is framed as adding to or subtracting from the final score rather than acting as a separate post hoc narrative.

The central idea is not the attribution method itself but the additive structure. When that structure holds, practitioners can compare contributions across features, inspect directionality, and reason about whether a prediction was driven by a small number of strong signals or a broader combination of weaker ones.

Why SHAP and LIME are both additive methods

SHAP and LIME both fit the additive attribution pattern, but they arrive there through different assumptions and estimation strategies. SHAP is designed around Shapley-value style contribution accounting, while LIME approximates a local explanation by fitting a simpler surrogate model near the prediction of interest.

That distinction matters because two methods can both be additive yet produce different governance outcomes. A model risk reviewer may prefer one method for consistency and theoretical grounding, while an analyst may prefer the other for speed, ease of use, or easier local interpretation.

Where additive attribution is useful

Additive feature attribution is most useful when the reader needs a decomposition of a specific prediction, not just a global summary of model behaviour. It is common in model explanation workflows, fairness review, exception analysis, and validation discussions where stakeholders want to see which inputs moved the score and by how much.

It also helps create a shared language between technical and non-technical reviewers. A contribution-based explanation is often easier to discuss than raw model internals, especially when the goal is to understand whether the model responded to the expected signals or an unexpected proxy.

Limitations and interpretation boundaries

Additive explanations are powerful, but they are still an approximation of model behaviour. The contributions depend on the chosen baseline, feature representation, sampling approach, and locality assumptions, so different setups can yield different answers even for the same underlying model.

That means the output should be treated as an explanation of the model under a chosen method, not as direct proof of causality. A plausible attribution can still be misleading if correlated features, unstable local neighbourhoods, or poor background data distort the estimate.

Risk and Threat Considerations

Additive feature attribution can create confidence without certainty if reviewers treat contribution scores as causal truth. The main security and governance risk is misinterpretation, especially when explanations are used to justify access decisions, fraud flags, safety decisions, or other high-impact outcomes.

Failure mechanism: Correlated inputs, baseline sensitivity, and surrogate-model simplification can make the explanation look precise while hiding instability or proxy behaviour. A misleading attribution can then pass review because it is easy to read, not because it is robust.

Impact: Organisations may approve flawed models, miss discriminatory or unsafe behaviour, or fail to notice when the explanation method is masking the true decision logic. That can undermine auditability, user trust, and the reliability of downstream governance decisions.

Practitioner Guidance

What to watch for: Treat additive attribution as a diagnostic tool, not a final verdict. When SHAP or LIME is used in a control, review, or exception workflow, validate that the chosen method is stable across reasonable perturbations and that the explanation is being interpreted at the right scope, local versus global.

Governance implication: Document which explanation method is approved for which decision type, because the choice between methods can change what reviewers believe the model is doing. Where decisions are sensitive, pair the explanation with independent validation rather than relying on the attribution output alone.

Related resources from NHI Mgmt Group

• Why do non-differentiable models complicate feature attribution in production AI systems?
• Why does feature attribution matter when evaluating machine learning predictions in regulated workflows?
• What breaks when feature attribution is treated as a truth source?
• Feature Attribution