Teams should choose based on the failure pattern. If the model is missing a capability because the training set is incomplete, add new labels. If the issue is incorrect ground truth or noisy annotations, fix the labels first. In production, prioritise the cases that change model behavior most, using error analysis and confidence signals to focus limited labeling effort where it matters most.
When to add new labels versus correct the training set
The right choice depends on whether the pipeline is failing because the model has not seen enough of the world, or because the existing labels are wrong. Adding more data helps when the gap is coverage. Fixing labels helps when the signal is contaminated. In practice, teams should separate “missing example” problems from “bad ground truth” problems before spending scarce annotation budget.
For unstructured ML work, that distinction matters because the same symptom, poor validation performance, can come from very different causes. A model that misses a class, style, domain, or edge case usually needs more representative labels. A model that appears inconsistent across similar examples often needs label review, annotation guideline cleanup, or reconciliation of conflicting sources. Those are different failures, and they call for different interventions.
When the issue is undercoverage, the best next step is to expand the labeled set around the blind spot. That is especially true when error analysis shows the model is systematically failing on a subpopulation, document type, language, layout pattern, or rare event that is absent or thin in the training data. The goal is not just “more data”, it is more of the data that changes model behaviour in the failing region.
When the issue is label noise, the priority is to repair the annotations before adding volume. Noisy labels can teach the model the wrong decision boundary, waste evaluation effort, and make retraining look worse than it should. If the same inputs receive different labels because the taxonomy is ambiguous or the annotators lack a shared rubric, adding more of that noise usually compounds the problem rather than fixing it.
How to tell whether the problem is coverage or label quality
Use error analysis to cluster the failures by pattern, not by instinct. If errors concentrate on cases the model rarely sees, treat the gap as a data acquisition problem. If errors cluster around examples that look semantically similar but carry conflicting annotations, treat the gap as a labeling quality problem. Confidence signals can help, but they should guide investigation, not replace it.
A useful test is to compare model mistakes against the annotation source. If the model is wrong but the label is also dubious, you likely have a ground truth issue. If the label is defensible and the model still misses it, the dataset may be too small or too narrow. In unstructured data, this often shows up in text, image, audio, or document parsing tasks where edge cases are unevenly represented and the labeling standard is hard to apply consistently. For broader pipeline hardening, teams often pair this review with AI Infrastructure Workload Identity Guide so training jobs, model registries, and related AI infrastructure remain observable and governed.
Another practical signal is disagreement rate. If annotators disagree because the task definition is fuzzy, you need label cleanup, clearer guidelines, or a taxonomy redesign. If annotators agree and the model still fails, you have stronger evidence that more representative labels are needed. The distinction saves teams from over-collecting data when the true issue is ambiguity.
For production prioritisation, focus first on the cases that change behaviour most. That means prioritising high-frequency errors, high-impact classes, and examples that expose systematic blind spots. A small amount of well-chosen labeling work usually beats a large batch of low-value annotation.
What good labeling strategy looks like in production
Good practice is iterative. Start with a small diagnostic slice, classify the failure mode, then decide whether to expand the set, relabel the set, or do both. Keep the annotation rubric stable while you test the model, because changing labels and model features at the same time makes it hard to learn which change actually improved performance.
Use a sampling plan that is deliberate rather than random. Target uncertain predictions, recently changed content, rare classes, and examples from segments where business impact is highest. If a model failure is concentrated in a critical subset, more labels from that subset can produce a larger gain than broad retraining on the whole corpus. For teams managing ML platforms at scale, AI platform identity and pipeline security also matters because training and labeling systems often depend on sensitive credentials and service access.
When the label set itself is unstable, treat taxonomy maintenance as part of model maintenance. If annotators cannot distinguish two classes reliably, the model usually cannot either. In that case, collapsing classes, rewriting the rubric, or introducing hierarchical labels may be more effective than forcing finer-grained annotation. If the taxonomy is sound, then more representative labels are the better investment.
Risk and Threat Considerations
Label quality problems can create durable model error, hidden bias, and false confidence in validation results. In unstructured ML pipelines, that is especially risky because noisy or inconsistent annotations can be hard to detect once they are embedded in large datasets and reused across training runs.
Failure mechanism: Incorrect ground truth, ambiguous taxonomy, or inconsistent annotator judgment teaches the model the wrong pattern, while incomplete coverage leaves important behaviours unlearned. Both failure modes can look like simple underperformance, which delays the correct fix.
Impact: Teams may waste labeling spend, retrain on the wrong signal, and ship models that fail in the exact cases that matter most, especially rare but high-impact cases.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5, NIST CSF 2.0, OWASP SAMM and OWASP ASVS set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | CM-3 — Configuration Change Control | Label taxonomy and dataset changes need controlled review to avoid unstable training data. |
| SI-10 — Information Input Validation | Noisy labels are bad input to the training process and must be validated before use. | |
| AU-6 — Audit Review, Analysis, and Reporting | Error analysis and confidence-driven triage rely on review of model and annotation outcomes. | |
| Recommendation — Apply CM-3 to review and approve dataset and rubric changes before retraining. Apply SI-10 to validate annotation inputs and reject inconsistent labels. Apply AU-6 to analyze model errors and prioritize corrective labeling work. | ||
| NIST CSF 2.0 | ID.RA-01 — Asset vulnerabilities are identified and documented | Training-data gaps and label defects are risk conditions that must be identified. |
| ID.RA-03 — Cyber threat and vulnerability information is received from information sharing forums and sources | Ongoing error signals and incident-like failures inform where labeling work is needed. | |
| Recommendation — Identify dataset coverage gaps and label-quality defects before choosing a remediation path. Use observed failure patterns and feedback to guide the next labeling cycle. | ||
| OWASP SAMM | IM — Implementation Management | Annotation and retraining loops are process controls that should be managed deliberately. |
| Recommendation — Manage annotation changes as a controlled lifecycle activity, not an ad hoc fix. | ||
| OWASP ASVS | V15 — Secure Coding and Architecture | The answer depends on disciplined system design and feedback loops in the ML pipeline. |
| Recommendation — Design the pipeline so labeling, evaluation, and retraining decisions remain separable and testable. | ||
Practitioner Guidance
What to prioritise: Start with error analysis on the highest-value failures, then separate missing coverage from label noise before expanding the labeling programme. If the same failure pattern repeats across clean labels, add data; if the labels themselves are unreliable, fix the rubric and relabel first.
What to verify: Check annotator agreement, class definitions, and whether the validation set reflects the production slice that is actually failing. If a class cannot be labeled consistently by humans, the model will not reliably learn it from more examples.
Practitioner takeaway: The best teams do not ask “more labels or better labels?” in the abstract, they ask which failure mode is driving the error and invest where the next annotation step will change model behaviour most.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 26, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
