Why do non-accountable fraud models create higher financial risk for merchants even when they appear to work?

Why This Matters for Security Teams

Merchants do not pay for fraud models in the abstract, they pay for the decisions those models make. A model can look strong if it reduces chargebacks, but that can hide a bad incentive structure: the model is not absorbing the downside when it over-declines legitimate buyers, slows checkout, or misses evolving fraud patterns. The financial exposure sits with the merchant, not the model owner.

The practical problem is that apparent success can be measured with the wrong target. If the only score being watched is chargeback reduction, the model may still be increasing false declines, customer support cost, and revenue volatility. That is why fraud performance has to be judged against merchant economics, not just model precision. In payment-heavy environments, PCI DSS v4.0 is also relevant because it reinforces least privilege and tight control over system and application accounts, which are often part of the fraud decisioning and payment stack. PCI DSS v4.0, PCI Security Standards Council

In practice, many teams discover this only after a “better” fraud model has already raised abandonment and margin pressure.

How It Works in Practice

Non-accountable fraud models usually optimise a proxy outcome, such as chargeback rate, fraud approval rate, or a generic classification score. That can be useful, but it creates a structural gap when the model is not tied to the merchant’s full profit-and-loss picture. A model that declines too aggressively may reduce fraud losses while quietly destroying good orders, and a model that is too permissive may preserve conversion while exposing the merchant to downstream fees, disputes, and manual review burden.

The key issue is accountability. When a model is not financially exposed to the consequences of its own errors, it has no built-in reason to balance false positives against false negatives in the way a merchant must. That usually leads to one of three failure patterns:

• Over-automation, where the model suppresses chargebacks but also rejects high-value legitimate customers.
• Under-adaptation, where the model stays “stable” by resisting change even as fraud patterns shift.
• Metric drift, where teams optimise a technical score that no longer reflects merchant profit, customer retention, or operational load.

Merchants should therefore treat the model as one control in a broader decision system, not as the decision maker itself. The real question is whether the fraud policy improves net revenue after chargebacks, fees, manual review, customer friction, and repeat-purchase impact are included. For financial institutions and payment-adjacent merchants, operational resilience and third-party dependency also matter because decisioning tools often sit inside wider ICT and outsourcing chains. EU Digital Operational Resilience Act (DORA)

These controls tend to break down when the fraud model is tuned in isolation from the merchant’s loss ledger and customer conversion data.

Common Variations and Edge Cases

Tighter fraud control often increases operational friction, so organisations have to balance fraud suppression against conversion loss and support overhead. That tradeoff becomes sharper in card-not-present commerce, subscription billing, and high-value digital goods, where a single bad policy can affect a large share of legitimate traffic.

There is also no universal standard for the “right” fraud threshold. Some merchants can tolerate more false declines because their fraud rate is extreme, while others need a much more permissive posture because their margins depend on repeat purchase behaviour. Models trained on one channel or one geography often fail when copied into a different product mix, because customer behaviour, dispute patterns, and fraud tactics are not stable across markets.

Another edge case is model goodharting. Once fraud teams optimise too narrowly for a single metric, the model may appear to improve while the business gets worse. That is especially common when manual review queues, refund rates, and customer lifetime value are excluded from the evaluation. FinCEN becomes relevant when fraud analysis intersects with suspicious transaction reporting and broader AML escalation, because some patterns need to be investigated as financial crime signals rather than treated only as model tuning problems. FinCEN

For merchants, the hardest cases are the ones that look operationally successful in dashboards but quietly reduce profit over time.

Risk and Threat Considerations

Non-accountable fraud models create a financial governance risk because they can externalise error costs onto the merchant while appearing effective in control reporting. The danger is not only fraud loss, but also false-decline leakage, customer churn, and unstable forecasting when the model optimises the wrong objective.

Failure mechanism: When the model is rewarded for suppressing chargebacks or hits a narrow acceptance target, it can become overly conservative, under-reactive to fraud drift, or blind to business impact outside its immediate score. Attackers also benefit when defenders trust a model that looks “good enough” and stop revisiting thresholds, feature quality, and review logic.

Impact: Merchants absorb the direct financial loss, operational overhead, and demand erosion. Over time, the model can make the portfolio less profitable even as the fraud dashboard improves.

Practitioner Guidance

What to prioritise: Evaluate fraud models against net merchant outcome, not only chargeback suppression. Include false-decline cost, review cost, refund rates, repeat purchase impact, and seasonality so the model is judged on the economics it actually affects.

Decision rule: If a model cannot explain why it should be allowed to increase merchant loss in one area to reduce it in another, treat it as an optimisation aid, not an accountable control. That distinction matters because non-accountable models should never be the final authority on high-value or high-friction transactions.

What to verify: Require evidence that threshold changes were tested against holdout periods and that review queues, approval rates, and customer complaints were measured together. A model that “improves” one line item while degrading the rest is usually the wrong model for production.

Practitioner takeaway: The safest fraud programme is not the one with the lowest chargeback rate, it is the one whose model behaviour stays aligned with merchant profit, customer experience, and operational resilience.

Related resources from NHI Mgmt Group

• Why do AI models with tool access create security risk even when they are not autonomous?
• Why do opaque machine learning models create higher governance risk in financial services?
• Why do manual compliance processes create higher operational and fraud risk in financial services?
• Why do large language models still create risk even when they produce fluent and confident answers?