How should organisations reduce security risk when fine-tuning code generation models on mixed-quality training data?

Why Mixed-Quality Training Data Becomes a Security Problem

Fine-tuning code generation models on mixed-quality data can turn ordinary repository noise into a repeatable source of insecure suggestions. The issue is not only correctness, but also whether the dataset contains patterns that normalise poor validation, weak input handling, unsafe dependency use, or inconsistent coding discipline. That matters because the model can reproduce those patterns at scale, even when the training examples looked harmless in isolation. For organisations, the real exposure is that quality defects become model behaviour, then migrate into development workflows as if they were accepted practice. NIST Cybersecurity Framework 2.0 provides useful context for governing this as part of broader risk management rather than a one-off engineering cleanup. In practice, many teams discover the problem only after the model starts echoing low-grade code conventions that were never meant to be copied into production.

What Good Data Curation Looks Like Before Fine-Tuning

Security-focused curation starts with separating usable training material from code that should never be treated as a positive example. That usually means screening for obvious vulnerabilities, obsolete patterns, broken error handling, hardcoded secrets, unsafe deserialisation, and examples that pass compilation but fail basic security review. The goal is not to make every file perfect. It is to ensure the fine-tuning set reflects the standard you want the model to learn, rather than the worst habits present in the source estate.

A practical workflow usually has three layers. First, remove material that is clearly unsafe or low-value, such as duplicated snippets, dead code, or examples with known security defects. Second, validate the remaining dataset against internal coding rules, dependency policy, and language-specific secure coding guidance. Third, sample outputs after fine-tuning to confirm the model is not generalising from weak patterns that survived initial filtering.

• Use quality gates before training, not just after output review.
• Treat security review as part of dataset selection, not a separate audit step.
• Prefer curated, representative examples over larger volumes of noisy code.
• Check whether the model is learning style from the dataset or unsafe behaviour from it.

This guidance breaks down when organisations cannot distinguish authoritative code from copied, outdated, or context-free snippets, because the model then inherits ambiguity as well as syntax.

Where Mixed Data Needs Extra Caution

Tighter curation often reduces dataset size and diversity, requiring organisations to balance training breadth against security confidence. That tradeoff becomes sharper in multilingual codebases, legacy estates, or teams that rely heavily on internal examples, because the “best available” sample may still contain inconsistent security maturity. There is also a genuine consensus gap on how much imperfect code should be tolerated in fine-tuning sets: some teams accept limited noise if they can label or downweight it, while others exclude anything that could teach a dangerous habit.

Edge cases usually appear when the data mixes secure and insecure variants of the same pattern. In those cases, the question is whether the model will learn nuance or uncertainty. If the surrounding commentary, labels, or structure are strong, the model may still benefit from comparative examples. If not, the safer choice is to keep the insecure version out of training altogether and use it only for evaluation or red-team testing. That distinction matters because security risk rises when the model is exposed to examples that look valid but encode the wrong default behaviour.

Organisations should be especially careful with generated code, copied forum examples, and internal snippets that were never reviewed for production use. These sources are often useful for volume, but they are also where weak patterns tend to cluster. The practical limit of this guidance is that no amount of filtering can fully compensate for a dataset whose provenance is unclear.

Practitioner Guidance

What to prioritise: prioritise provenance and secure-quality screening before you optimise for dataset size. If a sample cannot be defended as a safe example of the behaviour you want the model to imitate, it should not be treated as training-ready.

What to verify: verify that your curation process distinguishes three states: acceptable training example, evaluation-only example, and exclude entirely. Teams often underestimate how much ambiguity comes from mixed internal repositories, where “working code” is not the same as “safe code.”

Decision rule: if insecure code is useful for testing the model’s resistance to bad patterns, keep it out of fine-tuning and place it in validation or adversarial review instead. The model should be trained on what you want repeated, not on what you want detected.

Practitioner takeaway: the security outcome depends less on having a large dataset than on making sure the model is trained against a clear, reviewable standard of acceptable code behaviour.

Related resources from NHI Mgmt Group

• How can organisations reduce the risk of secrets in AI training data?
• How should organisations reduce the security risk of ROT data in cloud and SaaS environments?
• How can organisations reduce the risk of source code, credentials, and regulated data leaking into generative AI tools?
• How should security teams reduce identity risk when employees use large language models with sensitive enterprise data?