They need both, but for different purposes. Runtime guardrails reduce live risk on production traffic, while pre-release evaluation decides whether a change should ship in the first place. Teams that skip pre-release evaluation usually discover problems only after deployment, which makes remediation slower and more disruptive.
Why the choice is not either-or
Runtime guardrails and pre-release evaluation solve different problems. Guardrails shape what the model is allowed to do during live traffic, while evaluation asks whether the change is safe enough to release at all. If you only rely on runtime controls, you are treating deployment as the first test, which is usually too late for high-blast-radius failures.
That split matters because LLM risk is not confined to a single layer. Some failures are prompt-level, some are behavior-level, and some are release-level, so the control that catches one class of issue often misses another. A strong program therefore uses evaluation to gate ship decisions and runtime controls to contain residual risk after release.
For teams comparing tooling, the most useful mental model is to treat runtime guardrails as a production safety layer and pre-release evaluation as a release qualification layer. NHIMG’s AI Security Platform Buyer’s Guide is useful here because it frames guardrails, red teaming, and vendor evaluation as complementary capabilities rather than substitutes.
What each control is good at catching
Pre-release evaluation is best at finding systematic failures before users are exposed. That includes prompt injection susceptibility, unsafe tool use, harmful outputs, policy gaps, and regressions introduced by model, prompt, retrieval, or agent changes. It is especially important when a release changes the model’s behavior, not just its configuration.
Runtime guardrails are best at limiting the impact of unknowns and edge cases in production. They can block disallowed outputs, constrain tool calls, enforce allowlists, rate-limit abuse, redact sensitive material, and route suspicious interactions for review. That makes them valuable when you cannot fully predict how the system will behave under novel inputs.
The practical distinction shows up clearly in attack and abuse scenarios. NHIMG’s ForcedLeak (Salesforce Agentforce) 2025 illustrates why runtime controls matter when an attacker can steer an agent through live inputs, while EchoLeak (Microsoft 365 Copilot) 2025 shows that some failures only become obvious once the system is exercised in a realistic environment.
How to use both without creating false confidence
The most common mistake is to treat guardrails as evidence that a model is ready to ship. That is backwards. A runtime filter can reduce blast radius, but it does not prove the underlying model, prompt chain, or tool policy is sound. Likewise, a good evaluation result does not guarantee safety in production if the workload, users, data, or tools differ from the test set.
Use evaluation to decide whether the change can leave the gate, then use guardrails to decide how much risk is tolerable after release. If evaluation finds a critical issue, fix the design before deployment. If production monitoring shows guardrail violations, treat that as evidence the release assumptions were incomplete, not as proof that the guardrail has failed.
For release engineering, this means measuring both pre-release defect rate and post-release interception rate. A stable system should show fewer severe issues at evaluation time, and the runtime layer should be catching only the residual cases that escaped testing. NHIMG’s AI Security Platform Buyer’s Guide is also relevant as a procurement aid because it helps separate tools that inspect a system before launch from tools that intervene during execution.
Risk and Threat Considerations
Skipping pre-release evaluation shifts discovery into production, where failures are more expensive, more visible, and harder to contain. Runtime guardrails can reduce harm, but they do not remove the operational and security cost of a flawed release, especially when an LLM can call tools, expose data, or generate actions at scale.
Failure mechanism: A model, prompt chain, or agent path that was never stress-tested in realistic conditions reaches production, where the first meaningful test comes from users or attackers.
Impact: The organisation absorbs slower remediation, wider blast radius, and higher likelihood of data exposure, misuse, or service disruption before the defect is understood.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI 600-1 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI 600-1 | Generative Artificial Intelligence Profile | Covers GenAI governance and pre-deployment testing for release decisions. |
| Recommendation — Use pre-deployment testing and incident handling to gate GenAI releases. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Runtime guardrails must constrain agent actions and privilege use during execution. |
| ASI02 — Tool Misuse | Evaluation and guardrails both address unsafe tool invocation paths in agentic systems. | |
| Recommendation — Restrict agent privileges and enforce action boundaries at runtime. Test and block unsafe tool calls before they reach production. | ||
| MITRE ATLAS | Adversarial ML Threat Matrix | Supports threat modelling and red-team evaluation for AI/ML attack paths. |
| Recommendation — Map likely AI attack paths and validate controls against them. | ||
| NIST AI RMF | AI Risk Management Framework | Directly fits the governance choice between pre-release evaluation and live risk controls. |
| Recommendation — Manage AI risk across govern, map, measure and manage activities. | ||
Practitioner Guidance
What to prioritise: Treat evaluation as the release gate and guardrails as the containment layer. If you must choose where to spend the first dollar, spend it on testing the exact behavior that would create business or security harm, then add runtime controls around the highest-consequence actions.
What to verify: Confirm that pre-release tests cover the same prompts, retrieval paths, tool permissions, and user classes the production system will actually see. If the test setup does not match the live environment, the result should be considered partial, not authoritative.
Practitioner takeaway: The safest operating model is to assume every LLM release will miss something, then use evaluation to catch what should never ship and guardrails to limit the damage from what still does.
Related resources from NHI Mgmt Group
- When should organisations use runtime guardrails instead of relying on pre-deployment testing?
- When should organisations prioritise runtime protection over pre-release checks?
- Should organisations rely on pre-release testing or runtime app shielding for mobile risk?
- When should organisations use runtime authorization for AI agents?