Notifications
Clear all
Topic starter
19/08/2026 3:28 pm
TL;DR: A 22B on-premise code model fine-tuned on 4,811 validated security triples reached 98.0% safe rate on C/C++ remediation and 85.1% on Python generation, statistically tying Claude Opus 4.8 on the remediation benchmark while showing the biggest gains came from dataset breadth, according to Symbiotic Security. The result shifts the conversation from model size to governance of training data, evaluation limits, and where secure code generation can run without exposing sensitive source.
NHIMG editorial — based on content published by Symbiotic Security: Reaching Frontier Parity in Security Code Remediation with an On-Premise 22B Model
By the numbers:
- SecFix-V3 reached a 98.0% safe rate on CyberSecEval for C/C++ remediation.
- The model improved by 18.5 points on C/C++ remediation versus the un-fine-tuned base model.
- Only 13% of organisations feel extremely prepared for agentic AI despite rapid adoption.
Questions worth separating out
Q: What breaks when secure code models are trained on weak or narrow datasets?
A: They learn fluent but unreliable remediation patterns, which can make benchmark scores look stronger than real security.
Q: Why do secure code benchmarks often overstate real-world safety?
A: Because static-analysis benchmarks only measure the vulnerability patterns the analyzer can detect.
Q: How should teams evaluate on-premise AI code assistants before deployment?
A: Test them against the languages, frameworks, and weakness classes present in your environment, then add functional validation and human review for high-risk code paths.
Practitioner guidance
- Validate remediation datasets before training Require schema checks, duplicate removal, parser validation, and leakage screening before any security code fine-tuning run.
- Separate benchmark scoring from production approval Use static-analysis safe rates only as a screening signal, then add functional tests and human review before allowing generated patches into sensitive repositories.
- Track vulnerability class coverage by language Measure whether your training or evaluation set covers the languages and CWE classes that matter most to your codebase, because narrow coverage can inflate confidence while hiding gaps.
What's in the full report
Symbiotic Security's full research covers the operational detail this post intentionally leaves for the source:
- The full dataset construction and validation pipeline, including leakage checks and near-duplicate filtering.
- Per-version training dynamics across V1 to V3, showing how data breadth changed model behaviour.
- Benchmark tables with matched confidence intervals for the frontier comparison.
- The full error analysis by CWE class, including where the model still regresses.
👉 Read Symbiotic Security's research on on-premise secure code generation and remediation parity →
On-premise code remediation models: are your controls keeping up?
Explore further
19/08/2026 4:11 pm
On-premise code generation is becoming an identity and governance problem, not just an AI model problem. Once a model can patch security-sensitive code inside the perimeter, it influences access to source, remediation workflows, and developer trust boundaries. That shifts control questions toward who can train, approve, and deploy the system, and how its outputs are governed in regulated environments. Practitioners should treat secure code models as part of the identity-aware software supply chain.
A question worth separating out:
Q: Should organisations prefer on-premise code models for sensitive source code?
A: They should consider them where code confidentiality, data sovereignty, or regulatory constraints make hosted inference unacceptable. The trade-off is that the organisation inherits more responsibility for training quality, evaluation, and runtime governance, so the deployment decision should be based on control maturity rather than model novelty.
👉 Read our full editorial: On-premise code models can reach frontier parity in remediation
