How do teams know if EDD is actually working?

What good evidence looks like for EDD

EDD should be judged by whether it changes decisions, not by whether it creates more alerts. A functioning programme produces risk ratings that move when new facts emerge, triggers escalation at the right thresholds, and leaves a defensible record of why a counterparty, customer, or relationship stayed approved, was restricted, or was exited.

The practical test is whether the process can absorb fresh evidence without becoming arbitrary. If the same issue repeatedly produces the same outcome regardless of the facts, the programme is frozen. If the outcome changes but the rationale is not recorded, the programme is not auditable. Good EDD creates traceable judgement under uncertainty, not just a score.

That also means teams should separate activity from effectiveness. High case volume, long questionnaires, or many triggers do not prove control quality if reviewers are not translating them into consistent decisions. The real signal is whether investigators can show a repeatable path from evidence to disposition, especially where risk is elevated or the relationship is unusual.

Where EDD usually fails in practice

EDD fails when evidence collection, review, and escalation are disconnected. Teams often gather more information than they can interpret, then fall back to habit, manager preference, or inherited settings. That produces a system that looks busy but does not improve judgement. In that state, even strong alerts can become noise if they never alter the underlying decision.

Another failure mode is inconsistency across reviewers or business units. If one team escalates on a pattern that another team routinely approves, the issue is usually not the alert itself, but unclear criteria, weak ownership, or poor calibration. A healthy EDD process should narrow variation in outcomes for similar facts, while still allowing documented exceptions where the risk profile justifies them.

EDD also breaks when audit files are incomplete. A reviewer should be able to reconstruct what was known, what changed, who decided, and why the final action was proportionate. Without that chain, the programme may still detect risk, but it cannot prove that it managed risk in a controlled way.

How practitioners should measure EDD quality

Measure whether risk ratings are responsive, whether escalations are timely, and whether final decisions are explainable. Those three checks are more useful than raw alert counts because they assess the control outcome. A mature team can show that material evidence changes the case status, that exceptions are routed consistently, and that exits or restrictions are supported by documented reasoning.

Useful operating signals include reviewer agreement on similar cases, the share of escalations that lead to a changed decision, and the percentage of closed files with a clear rationale and supporting evidence. FATF Recommendations are a useful reference point when teams want to align diligence and escalation to risk-based judgement rather than checkbox completion. For control maturity, the same logic also maps well to NIST SP 800-53 Rev 5 Security and Privacy Controls and NIST Cybersecurity Framework 2.0, because both emphasise governance, auditability, and actionably managed risk.

Risk and Threat Considerations

When EDD is weak, the main risk is not that teams lack data, but that they produce a false sense of control. Excessive alerts with poor decisions create decision fatigue, missed escalation, and inconsistent acceptance of high-risk relationships. Over time, that can leave the organisation exposed to hidden counterparties, weakly justified exceptions, and failures that only show up after an adverse event.

Failure mechanism: Evidence is collected but not translated into consistent disposition logic, so reviewers either override risk casually or escalate without a stable basis. That allows weak relationships to remain open, or strong ones to be exited for the wrong reasons.

Impact: The programme loses credibility, auditability, and early-warning value. In practice, that means more operational work with less protection, and a higher chance that a material risk is missed until it has already affected the business.

Practitioner Guidance

What to verify: Pick a sample of escalated and non-escalated cases and confirm that a new fact would actually change the rating or disposition. If it would not, the workflow may be collecting evidence without using it.

What good looks like: Similar cases converge on similar outcomes, exceptions are documented with a reason, and the audit file shows the decision path in a way another reviewer could reconstruct.

Decision rule: If alerts are rising but escalation quality is flat, treat that as a control problem, not a tuning problem. Tighten the review criteria, clarify ownership, and test whether the team can defend the most difficult close calls.

Practitioner takeaway: EDD is working only when evidence changes decisions in a repeatable, explainable way, especially under higher risk, because that is where programme quality is actually visible.

Related resources from NHI Mgmt Group

• How do security teams know if Active Directory hardening is actually working?
• How do teams know if federated governance is actually working?
• How do teams know if identity security controls are actually working?
• How do security teams know whether least privilege is actually working?