Weights are the learned numerical parameters of a machine learning model. During training, optimisation adjusts them to reduce error, and during inference they determine how inputs are transformed into outputs, so changing weights changes the model's behaviour and, in some systems, the trust placed in its results.
What the term weights means in machine learning
Weights are the model’s learned parameters, so they are not just stored numbers. They encode the relationships the model has absorbed during training, and they are what turn the same input into different predictions when the model’s behaviour changes.
How weights are learned and updated
During training, the optimisation process adjusts weights to reduce error. That adjustment is iterative: the model makes a prediction, compares it with the target, and then updates the weights so the next prediction is closer. In effect, weights are the mechanism by which experience becomes model behaviour.
This is why small changes in weights can have outsized effects. In large models, many weights act together, so a slight shift in one layer can alter feature detection, ranking, classification, or generation downstream.
Why weights matter at inference time
At inference, the weights are fixed and define how incoming data is transformed through the network. They determine which patterns the model amplifies, which it suppresses, and how confidently it responds. If the weights are different, the model is effectively a different model, even when the architecture stays the same.
That also means weights are part of the model’s trust boundary. The outputs a user sees depend on the assumption that the weights are intact, authorised, and consistent with the trained version that was validated.
Weights, model integrity, and operational control
Because weights shape behaviour directly, they deserve the same integrity attention as the surrounding training and deployment pipeline. Teams need to know where the canonical weight files live, who can modify them, how versions are promoted, and how changes are detected. If weights are altered unexpectedly, the model can degrade, drift, or behave in ways that were never tested.
Weights are also one of the most important artefacts to protect when a model is shared, fine-tuned, or deployed across environments. The security question is not only whether the model runs, but whether it is the same model that was reviewed, approved, and monitored.
Risk and Threat Considerations
Weights are a high-value target because they can be changed to shift model behaviour without changing the code around the model. That creates integrity risk, supply-chain risk, and the possibility of subtle backdoored or degraded outputs that are hard to spot in normal testing.
Failure mechanism: An attacker, insider, or compromised pipeline can replace, poison, or tamper with weight files so the model appears legitimate but produces altered decisions, unsafe responses, or attacker-favoured outputs.
Impact: The result can be silent model compromise, loss of trust in predictions, downstream business errors, and a difficult post-incident recovery because the problem sits inside the model state rather than in an obvious application defect.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | SI-7 — Software, Firmware, and Information Integrity | Protects model weights as integrity-critical production artefacts |
| CM-5 — Access Restrictions for Change | Controls who may alter model files and deployment inputs | |
| AU-6 — Audit Record Review, Analysis, and Reporting | Supports review of unexpected model or artefact changes | |
| Recommendation — Verify model artefacts and alert on unauthorized weight changes. Restrict write access to approved model weights and related pipelines. Monitor and review parameter changes, promotions, and drift-related events. | ||
| CIS Controls v8 | CIS-3 — Data Protection | Covers protecting sensitive model artefacts and training outputs |
| CIS-8 — Audit Log Management | Supports traceability for changes to model artefacts and pipelines | |
| Recommendation — Protect trained weights and checkpoints with strong storage and transfer controls. Log and retain model update activity, promotions, and integrity checks. | ||
Practitioner Guidance
Why practitioners should care: Treat weights as controlled production artefacts, not as disposable training by-products. If your review process focuses only on code and ignores model parameters, you may miss the most direct path to behavioural compromise.
What to watch for: Unexpected accuracy shifts, unexplained output changes, or weight artefacts that do not match the approved training run should trigger review. A model can look healthy at the application layer while the underlying parameters have been altered.
Practitioner takeaway: The safest mental model is simple: code defines capability, but weights define behaviour.
Related resources from NHI Mgmt Group
- Why do runtime data sources matter as much as model weights in AI security?
- Why do poisoned chat templates matter if the model weights are unchanged?
- When should organisations choose local NVMe, shared file storage, or object storage for model weights?
- How should security teams evaluate the risk of deploying a new open-weights reasoning model in production?
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on November 16, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org