Notifications
Clear all
Topic starter
10/06/2026 10:43 pm
TL;DR: Risk scoring models are becoming harder to explain as fraud patterns evolve, and SumSub says that widening gap between performance and documentation now creates compliance exposure for regulated firms. The practical issue is not accuracy alone, but whether the model can survive regulatory scrutiny without turning into a legacy liability.
NHIMG editorial — based on content published by SumSub: a whitepaper on risk scoring, explainability, and model decay
Questions worth separating out
Q: How should compliance teams govern black box risk scoring models?
A: Compliance teams should require explainable decision traces, documented inputs, and a clear override path for every material model outcome.
Q: Why do risk scoring models become harder to trust over time?
A: Risk scoring models become harder to trust when data drift, fraud adaptation, and manual overrides accumulate faster than governance updates.
Q: What do regulators expect from AI and machine learning risk models?
A: Regulators generally expect transparency, documentation, and defensible decision logic, especially when a model influences access, onboarding, or fraud controls.
Practitioner guidance
- Map decision evidence to each score path Record the key inputs, thresholds, and override reasons for every material risk decision so reviewers can reconstruct why the model acted as it did.
- Track drift as a control signal Monitor changes in input distributions, override volume, and false-positive rates together, because each can indicate that the model has moved away from its documented operating assumptions.
- Separate model performance from governance approval Require an explicit review layer that can challenge, approve, or suspend model use when documentation quality no longer supports the current risk decisioning workflow.
What's in the full report
SumSub's full whitepaper covers the operational detail this post intentionally leaves for the source:
- The 10-point framework for surviving a regulatory deep-dive on model transparency and evidence quality.
- Practical methods for making AI and machine learning models explainable without reducing detection capability.
- The specific triggers, including manual overrides and data drift, that can turn a strong model into a legacy liability.
- Industry-focused implications for financial services, crypto, igaming, and trading teams that need to defend automated risk decisions.
👉 Read SumSub's whitepaper on explainable risk scoring and model decay →
Black box risk scoring: what it means for compliance teams?
Explore further
11/06/2026 2:51 am
Model opacity is now a governance failure mode, not just a data science limitation. When a risk score cannot be explained with enough evidence for review, the organisation loses control over the decision boundary itself. That weakens auditability across fraud, access, and compliance workflows, because reviewers cannot tell whether the model is behaving as designed or drifting into ungoverned judgment. Practitioners should treat explainability as part of control design, not a reporting afterthought.
A few things that frame the scale:
- 97% of NHIs carry excessive privileges, increasing unauthorised access and broadening the attack surface, according to the Ultimate Guide to NHIs.
- Only 5.7% of organisations have full visibility into their service accounts, which leaves most access decisions difficult to review or evidence.
A question worth separating out:
Q: When should a risk model be revalidated or retired?
A: A model should be revalidated when overrides rise, input patterns drift materially, or reviewers can no longer defend the documented logic with current evidence. It should be retired if its control story no longer matches operational reality. The right trigger is not age alone, but whether the governance record still supports trust.
👉 Read our full editorial: Black box risk scoring and model decay are raising compliance risk
