Provenance controls should come first when unverified models, datasets, or dependencies can enter production, because you cannot safely monitor what should never have been trusted. Runtime monitoring still matters, especially for agents and inference endpoints, but it is a second line of defence. The right sequence is trust establishment before operational expansion.
Why provenance comes first in the AI control stack
Provenance controls answer a simpler but earlier question than runtime monitoring: what should be allowed into the trust boundary at all? If model weights, training data, packages, prompts, or agent dependencies are unverified, the organisation is already accepting unknown behaviour into production. That makes provenance a prerequisite for any meaningful control strategy, not a substitute for it.
This sequencing matters because runtime monitoring only observes what is already executing. It can detect drift, abuse, or anomalous outputs, but it cannot restore trust in a component whose origin, integrity, or dependency chain was never established. For build and release chains, that logic aligns with provenance and integrity controls such as SLSA, which is why provenance should be treated as the first trust gate for AI systems.
In practice, provenance also reduces the scope of what monitoring has to watch. When you can prove where an artefact came from and what entered the release path, runtime detection becomes a second layer that focuses on behaviour, not basic trust establishment. That is a materially better operating model than trying to compensate for untrusted inputs with telemetry alone.
What runtime monitoring is good at, and what it cannot replace
Runtime monitoring is the right control for post-deployment visibility: it can surface unusual tool calls, abnormal output patterns, unsafe API consumption, prompt injection effects, or agent behaviour that diverges from the intended operating envelope. For AI endpoints and agentic systems, this is essential because good provenance does not prevent all misuse, especially when the model interacts with live systems.
But monitoring is inherently reactive. It depends on a baseline, logging, alerting, and an operator who can interpret what changed. If the underlying model, dataset, or dependency chain was never vetted, the defender may be monitoring a compromised or untrustworthy system from the start. That is why runtime controls belong after provenance, not before it. The same logic is reflected in guidance for AI risk governance such as NIST AI Risk Management Framework and ISO/IEC 42001:2023 AI Management System Standard, both of which assume controlled lifecycle governance rather than blind operational trust.
Runtime monitoring becomes especially important once agents have tool access, because behaviour can change after deployment even when the original artefact was trustworthy. In that sense, monitoring is the control that catches drift, abuse, and unsafe action, while provenance is the control that keeps unauthorised trust from entering the environment in the first place.
How to sequence both controls without creating a false choice
The practical answer is not either-or. Organisations should start by establishing provenance requirements for the artefacts and dependencies that can reach production, then add monitoring for the runtime behaviours that remain risky after trust is established. That means release gating, signed artefacts, dependency verification, and explicit approval of model sources first, followed by observability, alerting, and intervention paths for live systems.
A useful way to think about the sequence is: provenance decides whether the system is allowed to exist in production, while monitoring decides whether the system is behaving as expected once it is there. If the first step is weak, the second step is forced to carry too much weight and becomes a detective control trying to compensate for a preventable trust failure. Supply-chain and deployment controls such as NIST SP 800-190 Container Security and SLSA are useful reference points for that sequencing.
For AI programmes, the strongest pattern is to make provenance a release criterion and monitoring an operations criterion. That keeps the trust decision at the point where it is most effective, and reserves monitoring for the conditions where it adds real defensive value: behavioural deviation, abuse, and unexpected system interaction.
Risk and Threat Considerations
When provenance is deferred, organisations increase the chance that malicious, altered, or simply unknown artefacts can be promoted into production and then operated as if they were trusted. The threat is not only outright compromise, but also dependency confusion, poisoned datasets, tampered model artefacts, and unsafe third-party components that make downstream behaviour unreliable.
Failure mechanism: Unverified inputs enter the build or deployment path, and runtime monitoring is asked to detect problems after trust has already been granted. That weakens containment because the control is operating late, against a system whose baseline may already be invalid.
Impact: The organisation can inherit hidden model behaviour, unsafe tool use, policy bypass, data leakage, or persistent operational exposure, and it may only discover the issue after the system has already influenced decisions or taken actions at scale.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
SLSA, NIST AI RMF and NIST SP 800-190 set the technical controls, while ISO/IEC 42001:2023 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| SLSA | Supply-chain Levels for Software Artifacts | Provenance and integrity of AI artefacts depend on trusted build and release chains. |
| Recommendation — Require signed, traceable artefacts before promotion to production. | ||
| NIST AI RMF | AI Risk Management Framework | The question is about sequencing AI risk controls across governance and operations. |
| Recommendation — Establish provenance before relying on runtime monitoring for AI risk. | ||
| ISO/IEC 42001:2023 | AI Management System Standard | AI management systems need lifecycle governance, accountability, and controlled deployment. |
| Recommendation — Define release and monitoring controls inside the AI management system. | ||
| NIST SP 800-190 | Container Security | Runtime AI services still need operational controls after trusted deployment. |
| Recommendation — Harden the runtime environment and monitor deployed services continuously. | ||
Practitioner Guidance
What to prioritise: Put provenance requirements on the shortest path to production, especially for models, datasets, prompts, weights, packages, and agent dependencies that can change system behaviour. If you cannot verify origin and integrity, treat the artefact as unfit for trusted deployment.
What to verify: Confirm that release gates actually block unsigned, unapproved, or untraceable artefacts, and that monitoring exists for the runtime behaviours that provenance cannot eliminate, especially tool use, external calls, and output anomalies.
Decision rule: If the question is “can this be trusted to enter production?”, provenance answers first; if the question is “is it behaving safely now?”, monitoring answers second.
Practitioner takeaway: The mature pattern is not to choose between provenance and monitoring, but to use provenance to establish trust and monitoring to police trusted execution after deployment.
Related resources from NHI Mgmt Group
- Should organisations prioritise provenance controls or runtime guardrails first?
- Should organisations prioritise runtime controls or privacy-preserving AI first?
- How do organisations decide whether to prioritise AI workflow controls or insider-risk monitoring?
- Should organisations prioritise external exposure or internal credential governance first?