On-premise code models can reach frontier parity in remediation

At a glance

What this is: This research shows that an on-premise 22B code model can reach frontier parity on security remediation when it is trained on validated vulnerability-patch data and evaluated with static analysis.

Why it matters: For IAM, NHI, and broader security teams, it matters because code generation and remediation workflows increasingly intersect with sensitive source, controlled environments, and governance over what data can leave the perimeter.

By the numbers:

• The model reaches a 98.0% safe rate on CyberSecEval for C/C++ remediation, measured by static analysis.
• The model reaches an 85.1% safe rate on SecurityEval for Python generation, measured by static analysis.
• The training set grows to 4,811 validated security triples spanning 12 programming languages.
• The study says dataset breadth moved from 502 to 4,811 training pairs across one to twelve languages.

Context

Security code generation is no longer just a model-quality problem. It is also a governance problem because remediation systems may need access to vulnerable source, proprietary codebases, and regulated environments that cannot tolerate external inference APIs. That shifts the evaluation question from raw capability to whether a self-hosted model can be trained and operated with defensible controls over data, workflow, and deployment boundaries.

This article sits at the intersection of AI security and secure software delivery. The central issue is not whether a model can generate plausible fixes, but whether the training data, validation pipeline, and runtime isolation are strong enough to make on-premise use credible for sensitive code. For security teams, that creates a practical decision about where secure coding assistance can run without weakening confidentiality or change control.

Key questions

Q: How should teams govern on-premise code models that remediate vulnerabilities?

A: Govern them as part of the software delivery control stack, not as a standalone AI tool. Teams should define where source code may be processed, what training data is permitted, how patches are validated, and which review gates remain mandatory before code reaches production. Local hosting helps with confidentiality, but it does not remove the need for clear assurance boundaries.

Q: Why can a security code model look strong while still being risky?

A: Because static-analysis pass rates only confirm that a known rule did not fire, not that the patch is functionally correct or that every weakness class is covered. A model can score well on detectable CWEs while still leaving unmeasured defects, regressions, or false confidence in place. That is why benchmark results need independent testing and human review.

Q: What breaks when remediation training data is too narrow?

A: The model overfits the patterns it has seen and fails when the target language or weakness class changes. In this study, the biggest gains came from moving from one to twelve languages and from 502 to 4,811 validated pairs, which shows that weak diversity produces brittle remediation behaviour. Narrow data narrows the model’s security usefulness.

Q: Should organisations choose self-hosted code remediation over hosted APIs?

A: They should choose based on data sensitivity, regulatory constraints, and review maturity. Self-hosted deployment is attractive when source code cannot leave the perimeter, but it only works if the organisation can support high-quality training data, secure inference infrastructure, and independent validation of output quality. Otherwise, hosted and self-hosted both remain assurance problems, just in different places.

Technical breakdown

Why validated patch triples matter for secure code generation

Security remediation models depend on preference data that teaches the model to choose a safe patch over a plausible unsafe one. In this study, each example is a triple of vulnerable code, correct patch, and incorrect patch, which gives the model a direct safety signal rather than a generic coding signal. The important point is that the model is not simply learning syntax or style. It is learning to suppress patch patterns that preserve the underlying weakness, which is why validation quality matters more than raw corpus size.

Practical implication: Treat remediation data as a security control surface, not just training content, and validate that each example actually distinguishes safe from unsafe behaviour.

How on-premise fine-tuning changes the deployment boundary

On-premise fine-tuning changes where source code and model traffic live. Instead of sending code to a hosted API, the full training and inference loop stays inside the environment, here on a single 128 GB unified-memory device with no external inference calls during serving. That matters in regulated, defense, and proprietary settings where data exfiltration risk and contractual restrictions block cloud use. The architectural question becomes whether the self-hosted stack can maintain enough evaluation rigor to justify the reduced exposure.

Practical implication: Use locality as a governance constraint, but pair it with strict dataset, inference, and logging controls so perimeter residency does not become an assurance shortcut.

Why dataset breadth outperformed parameter tuning

The paper’s most important finding is that remediation quality improved most when the dataset grew broader, not when hyperparameters became more elaborate. Moving from 502 to 4,811 validated pairs and from one to twelve languages produced the clearest gains, while an overly strong DPO setting regressed performance. That shows secure code models are sensitive to coverage, language variety, and preference signal cleanliness. In practice, remediation capability is constrained by what the model has seen and how cleanly the safe and unsafe examples were separated.

Practical implication: Prioritise diverse, validated security examples over aggressive tuning if you want measurable gains in code-remediation quality.

NHI Mgmt Group analysis

Secure remediation is now a governance problem, not just a model benchmark. Once code generation is used to fix vulnerabilities, the control question shifts to who can train on sensitive source, where the model runs, and how safety claims are validated. That is directly relevant to security teams running software delivery, because the same workflow can reduce risk or spread it depending on the quality of data governance. Practitioners should treat remediation models as part of the secure development control plane.

Dataset breadth is the real control variable in code-remediation quality. The strongest gains came from expanding validated examples across languages and weakness classes, not from marginal changes to optimisation. That means model assurance depends on curation discipline, not just model selection. For the field, the lesson is that secure code AI will plateau quickly unless training data reflects real vulnerability variety and negative examples are cleaned with care. Practitioners should invest in data engineering as a security function.

On-premise deployment reduces exposure, but it does not create trust by itself. Keeping inference inside the perimeter helps when source code cannot leave the environment, yet the study also shows that static-analysis safe rates remain an upper bound, not proof of secure behaviour. This is the kind of gap that matters in AI governance and in application security. Teams should assume that local execution narrows exposure while leaving validation accountability fully in place.

Static-analysis-driven claims need a named concept: verification blind spot. The article demonstrates that a model can look strong under rule-based scanning while still leaving unmeasured weaknesses or functional regressions unresolved. That blind spot is especially relevant when organisations use AI to accelerate remediation and then over-trust the benchmark. The practitioner conclusion is simple: pair code-generation programmes with independent functional review and targeted security testing.

Self-hosted code models are becoming a viable option for sensitive environments, but only when the surrounding controls are mature. The market signal is not that cloud models are obsolete; it is that self-hosted remediation is now plausible for organisations that can support validated training data, reproducible evaluation, and deployment isolation. That aligns with broader secure software and AI governance trends. Practitioners should reassess where code assistance belongs in high-sensitivity pipelines.

From our research library:

• The average time to mitigate a leaked secret is 36 hours, highlighting the operational burden of manual remediation processes, according to the 2024 State of Secrets Management Survey.
• The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to the State of Secrets in AppSec.
• Read next: AI Coding Agents Security Guide

What this signals

Verification blind spot: security-code benchmarks can overstate assurance when they rely on static rules that only detect a subset of weakness classes. Teams that operationalise AI-assisted remediation need a second line of functional and security testing, otherwise a high safe rate becomes a false proxy for secure output.

Dataset breadth is the determining variable: this article shows that expanding validated examples across languages produced more improvement than tuning alone, which means remediation quality is now a data-governance issue as much as a model-selection issue. The operational lesson is to invest in curation, coverage, and leakage control before chasing marginal model changes.

Manual remediation still takes time after a secret leak, with the average time to mitigate a leaked secret at 36 hours according to the 2024 State of Secrets Management Survey, which is why automation claims need governance and not optimism.

For practitioners

• Validate remediation datasets before fine-tuning Require every training example to contain a vulnerable snippet, a correct patch, and a plausible rejected patch, then remove near-duplicates and cross-source overlap before training.
• Run secure code models inside the regulated boundary Keep inference and data preparation on systems that do not send source code to external APIs, especially where proprietary code or regulated workloads are involved.
• Separate static-analysis scores from security assurance Treat scan pass rates as an upper bound, then add functional testing and targeted review for weakness classes the analyzer does not cover well.
• Track coverage by language and weakness class Measure whether remediation gains come from broader language coverage and balanced CWE representation rather than from a single benchmark improvement.

Key takeaways

• On-premise remediation models can approach frontier performance, but the result depends on validated data, not just larger weights or better tuning.
• The study’s strongest gains came from broader language coverage and cleaner preference data, which makes training governance a security control.
• Static-analysis safe rates are useful signals, but they do not replace functional verification or review of weakness classes outside the scanner’s reach.

Key terms

• Security Remediation Model: A security remediation model generates code fixes for known vulnerabilities rather than generic code completions. Its quality depends on how well it learns the difference between a safe patch and a plausible patch that still preserves the flaw.
• Preference Optimization: Preference optimization trains a model to prefer one response over another based on paired examples. In security code generation, that means teaching the model to rank the correct patch above an unsafe but believable alternative.
• Static-analysis safe rate: Static-analysis safe rate is the percentage of generated code that does not trigger the chosen rule set in a scanning tool. It is a practical benchmark measure, but it does not prove functional correctness or complete security because it only reflects the weaknesses the scanner can detect.
• On-Prem Inference: On-prem inference means the model processes queries and context inside the organisation's own environment rather than sending them to an external service. For identity security, that reduces exposure of cloud configuration, risk scores, and threat context while keeping the reasoning layer inside the trust boundary.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore nhimg.org for resources that connect identity governance to the broader security disciplines your programme depends on.