Because ML learns patterns from the data it sees, and flawed data changes what the model can learn. Duplicates can over-weight examples, conflicting labels create ambiguity, and invalid values can teach impossible relationships. In validation, the same issues can make metrics look better or worse than reality, which breaks decision-making.
Why data quality changes what ML can actually learn
ML systems are not reading ground truth, they are inferring patterns from the training and validation data you give them. That means data quality is not a housekeeping issue, it is part of the model’s learning signal. When the input is noisy, duplicated, incomplete, inconsistent, or mislabeled, the model can learn the wrong associations, miss the real ones, or become overconfident in patterns that do not generalise.
Duplicates matter because they increase the effective weight of repeated examples, which can bias the model toward a subset of the data. Conflicting labels matter because they create ambiguity in the target the model is trying to optimise. Invalid or impossible values matter because they teach the model relationships that cannot exist in production, which often shows up later as brittle behaviour.
How bad data distorts validation metrics
Metrics are only as trustworthy as the evaluation data behind them. If the validation set contains the same quality issues as training data, metrics can look artificially good or misleadingly bad. Duplicate records can leak the same pattern into both training and validation, inflating apparent performance. Label noise can suppress measured accuracy even when the model is learning the right trend. Outliers, missing values, and broken feature definitions can also shift precision, recall, calibration, and loss in ways that do not reflect reality.
This is why a model can look strong in offline testing and then disappoint in production. The metric may be technically correct for the flawed dataset, but still wrong for the business decision. In practice, the evaluation process must be treated as a data-quality problem as much as a modelling problem.
Why data problems create business risk, not just model noise
Data quality problems do more than reduce benchmark scores. They can change which cases get prioritised, which users get flagged, or which thresholds teams choose to deploy. For example, if bad labels systematically underrepresent an error class, the model may miss the very outcome the organisation cares about. If training and validation data are contaminated in the same way, teams can approve a model that appears stable until it is exposed to cleaner or more diverse real-world data.
For teams that use ML in operational decisions, this is a governance issue as well as a technical one. The model may be behaving consistently with the flawed data, but the organisation is still making decisions from a distorted measurement system.
Risk and Threat Considerations
Data quality failures create a reliability risk because they can hide model weakness during testing and then surface as poor decisions after deployment. In adversarial settings, deliberately injected duplicates, label corruption, or malformed records can also be used to skew model behaviour or evaluation results.
Failure mechanism: contaminated training or validation data changes the statistical signal the model sees, so the optimisation target no longer matches the real-world problem.
Impact: the model can learn biased, unstable, or non-generalising patterns, and the reported metrics can overstate confidence in a system that will fail when it meets clean production data.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0, NIST SP 800-53 Rev 5, OWASP ASVS and SLSA set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | ID.AM-01 — Physical devices and systems within the organization are inventoried | Data quality evaluation depends on trustworthy asset and dataset inventory. |
| PR.DS-01 — Data-at-rest is protected | Dataset integrity and tamper resistance are central to reliable ML training and validation. | |
| DE.CM-09 — Computing hardware and software, data, and records are monitored for anomalies | Detecting duplicates, corruption, and pipeline anomalies supports trustworthy ML metrics. | |
| Recommendation — Inventory datasets and data pipelines so training and validation inputs are traceable. Protect dataset integrity so training and validation data cannot be silently altered. Monitor data pipelines for anomalies that can distort model metrics or learning. | ||
| NIST SP 800-53 Rev 5 | SI-10 — Information Input Validation | Invalid values and malformed records directly distort model inputs and evaluation data. |
| AU-6 — Audit Record Review, Analysis, and Reporting | Reviewing data and pipeline logs helps detect contamination that skews metrics. | |
| Recommendation — Validate inputs before they enter training or scoring pipelines. Review pipeline evidence to spot duplicated, conflicting, or corrupted records. | ||
| OWASP ASVS | V15 — Secure Coding and Architecture | Model pipelines are software systems whose correctness depends on controlled data flows and validation. |
| Recommendation — Build data validation and provenance checks into the pipeline architecture. | ||
| ISO/IEC 27001:2022 | A.8.9 — Configuration management | Versioned datasets and controlled pipeline changes are essential to repeatable ML evaluation. |
| Recommendation — Version datasets and pipeline settings so metric changes are explainable. | ||
| SLSA | Supply-chain Levels for Software Artifacts | Dataset and pipeline provenance matter when judging whether ML results are trustworthy. |
| Recommendation — Apply provenance discipline so data and artifacts used in ML can be trusted. | ||
Practitioner Guidance
What to verify: Check whether training, validation, and test data follow the same definitions for labels, timestamps, feature derivation, and filtering. If any split was built from the same upstream pipeline without quality controls, assume the metric may be optimistic until proven otherwise.
Common mistake: Treating accuracy as the main signal while ignoring whether the dataset itself is representative, deduplicated, and consistently labelled. A clean-looking score on flawed data is a measurement problem, not proof of a good model.
What good looks like: Stable metrics across clean, versioned datasets; explicit handling of duplicates and missingness; and a documented data quality review before model acceptance. The key judgement is to validate the data generating process, not just the model output.
Practitioner takeaway: When ML performance changes unexpectedly, the first question is often not “is the model broken?” but “is the data telling the truth?”