A pre-training safety assessment is the formal review performed before a model is trained to judge whether it could enable serious harm. The assessment examines misuse potential, catastrophic failure modes, and whether the planned safeguards are strong enough to reduce unacceptable risk before development proceeds.
Why pre-training safety assessment exists
A pre-training safety assessment is the point where teams decide whether a planned model should proceed at all. It is not just a policy checkbox, it is the first structured chance to compare intended capability, likely misuse, and the strength of proposed mitigations before expensive training makes the decision harder to reverse.
This matters because pre-training is where assumptions are still malleable. If the model’s intended use, data sources, or safety posture already point toward unacceptable harm, the assessment should surface that early enough to change scope, add safeguards, or stop the build.
What the assessment examines
The assessment typically looks at three things together: misuse potential, catastrophic failure modes, and whether the planned safeguards are proportionate to the risk. That means reviewing both normal expected use and foreseeable abuse, then asking whether the design can still be justified if the model is scaled, adapted, or repurposed.
For safety teams, the useful question is not only “can it be trained?” but “what could this model enable once it exists?” A strong assessment considers harmful capability emergence, dual-use value, operational controls, human oversight, release constraints, and whether the training objective itself is already too risky to pursue.
How it differs from later-stage evaluation
Pre-training safety assessment happens before training, so it is fundamentally a go or no-go judgment. Post-training testing, red teaming, and deployment review can still find important issues, but they are downstream of the largest design choices and may be too late to avoid sunk cost or unintended capability creation.
That timing changes the security posture. Early review is better for stopping unsafe work, narrowing the problem definition, or requiring stronger safeguards before the model ever learns from the chosen data. Later evaluation is better for measuring the model you already have.
What good practice looks like
Good practice is a documented, cross-functional review that produces a defensible decision, not an informal approval. It should be specific about the intended model behavior, the worst credible misuse cases, the safeguards being relied on, and the conditions that would trigger redesign, escalation, or cancellation.
Because the assessment is about deciding whether the risk is acceptable before training, the bar should be evidence-based and conservative. If the safety case depends on vague promises, undefined monitoring, or controls that have not been shown to reduce the relevant harm, the assessment has not really answered the question.
Risk and Threat Considerations
Pre-training safety assessment exists because harm can be built into a model before any public release. If the review is weak, teams can train systems that are easier to misuse, harder to constrain, and more costly to unwind after harmful capability has already been created.
Failure mechanism: The assessment misses a realistic misuse path or overestimates the effectiveness of proposed safeguards, so training proceeds with an unsafe objective, unsafe data, or an unsafe release posture.
Impact: The organisation may end up with a model that amplifies abuse, increases the scale of harmful output, or creates a capability that cannot be safely governed once training is complete.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST AI RMF and NIST CSF 2.0 set the technical controls, while ISO/IEC 42001:2023 define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | GOVERN — Govern | Pre-training safety assessment is an AI governance decision before model development proceeds. |
| MAP — Map | The assessment maps intended use, misuse potential and harm scenarios to the model context. | |
| MANAGE — Manage | The assessment manages risk treatment choices, safeguards and escalation before training. | |
| Recommendation — Define approval criteria and accountability before training starts. Map the model context and plausible harm pathways before build approval. Manage residual AI risk by requiring stronger safeguards or stopping the project. | ||
| ISO/IEC 42001:2023 | 8.2 — AI Risk Assessment and Treatment | This term is a pre-development AI risk assessment and treatment decision. |
| Recommendation — Perform AI risk assessment and treatment before authorising model training. | ||
| NIST CSF 2.0 | GV.RM-01 — Risk Management Strategy | A pre-training safety assessment supports organisational AI risk acceptance decisions. |
| ID.RA-01 — Risk Assessment | The assessment evaluates misuse and catastrophic failure modes before training. | |
| Recommendation — Set risk acceptance criteria for training decisions and escalate exceptions. Assess model misuse and harm scenarios before proceeding to training. | ||
Practitioner Guidance
Why practitioners should care: This is the control point where an unsafe model can still be stopped cheaply. Once training begins, the cost of changing direction rises fast, and weak early judgment can become a permanent downstream governance problem.
Common misunderstanding: A positive assessment does not mean the model is safe in all contexts. It means the current plan, as proposed, appears acceptable under stated assumptions, which must still be revisited as scope, data, and intended use change.
