Pre-deployment testing looks for weaknesses before an agent is released, using adversarial scenarios to expose unsafe behaviour. Runtime security protects the agent while it is operating, monitoring live inputs, outputs, and actions for abuse or drift. Organisations need both because some failures only appear under production conditions, especially when agents interact with real users and systems.
Testing the agent before release versus defending it while it runs
Pre-deployment testing and runtime security answer different questions. Testing asks whether the AI agent can be made to fail in a controlled evaluation, so teams can find unsafe tool use, weak instruction handling, prompt injection susceptibility, or policy gaps before users see them. Runtime security asks whether the same agent remains safe once it is exposed to live prompts, changing context, and real systems, where abuse, drift, and chained actions are more likely.
The difference matters because an agent that looks robust in evaluation can still behave differently under production pressure, especially when it can retrieve data, call tools, or act on behalf of a user. For that reason, pre-release validation should be treated as a readiness check, not proof of safety. The OWASP OWASP Top 10 for Agentic Applications 2026 is useful here because it separates design-time weaknesses from runtime abuse paths in a way practitioners can operationalise. In practice, many security teams discover the gap only after production inputs start steering agent behaviour in ways their test cases never covered.
How the two controls work across the agent lifecycle
Pre-deployment testing is usually evidence-led and scenario-driven. Teams build adversarial prompts, malformed tool requests, data exfiltration attempts, and unsafe workflow sequences to see whether the agent resists manipulation before release. The goal is to expose design weaknesses: poor prompt isolation, overbroad tool permissions, missing approval steps, weak content filters, or unsafe fallbacks. This stage is strongest when it uses realistic tasks, because synthetic happy-path testing often misses the exact combinations that create harm in production.
Runtime security starts where testing stops. It monitors the live system for indicators that the agent is being steered, overloaded, or misused. That can include input screening, output policy checks, tool-call validation, step-up approvals, anomaly detection, rate limiting, and logging that preserves enough context to reconstruct a harmful chain of actions. It is also the layer that can respond when model behaviour drifts over time, when external data changes, or when a user session becomes adversarial after initially looking legitimate. NIST’s NIST AI Risk Management Framework is relevant because it frames AI risk as a lifecycle issue rather than a one-time gate.
- Pre-deployment testing is strongest for finding known classes of weakness before exposure.
- Runtime security is strongest for detecting abuse that only emerges under real context, scale, or adversarial interaction.
- Testing can validate design assumptions; runtime controls can enforce them when assumptions fail.
- Neither layer should be treated as a substitute for the other if the agent can take actions outside its own interface.
The practical boundary is simple: if the question is “Can we safely ship this behaviour?”, testing is the lead control; if the question is “Can we keep this behaviour safe under live pressure?”, runtime security is the lead control. This guidance breaks down when the agent’s tool access is so broad that even a single missed failure path can become an immediate operational incident.
Where the boundary blurs and why that changes the answer
Tighter runtime control often increases friction for users and operators, so organisations have to balance safety against false positives, latency, and workflow interruption.
Some AI agent risks sit in both layers. Prompt injection, for example, can be tested before release, but its real-world variants often depend on live content, user-supplied files, or external web pages that were not present in the lab. Tool abuse is similar: pre-deployment testing can show that an agent is capable of unsafe actions, but runtime security must stop those actions when context changes or when a benign workflow turns malicious mid-session. The same is true for agent drift, where an evaluation may pass but production behaviour changes because the model, prompt stack, retrieval corpus, or connected tools have changed.
There is also a governance distinction. Pre-deployment testing is mainly about release confidence, while runtime security is about containment and accountability after release. For high-impact workflows, practitioners should not treat a strong test report as a green light unless the runtime layer can enforce boundaries on data access, tool use, and escalation. The most common mistake is assuming that red-team results and policy checks cover the same ground when they actually answer different risk questions. Anthropic’s report on an AI-orchestrated cyber espionage campaign is a useful reminder that agentic misuse can become operational only after the system is live and can chain actions over time. The runtime layer is what turns that lesson into a control decision rather than a retrospective finding.
Risk and Threat Considerations
AI agents create a dual exposure: design-time weakness and live operational abuse. A system that passes pre-deployment tests can still be exploitable once it is connected to real users, external content, and privileged tools, because the attack surface expands as soon as the agent starts acting in context.
Failure mechanism: Adversarial prompts, poisoned context, unsafe tool invocation, and session drift can bypass assumptions that were valid during testing. In practice, attackers do not need to break the model outright if they can steer its inputs, chain its actions, or exploit overbroad permissions after deployment.
Impact: The agent may disclose sensitive data, execute unintended actions, alter records, or propagate unsafe decisions across downstream systems. Where the agent has operational authority, a runtime failure can become a business process failure, not just a model-quality issue.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, MITRE ATLAS and CSA MAESTRO address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A1 — Agentic Prompt Injection | Directly addresses pre-release and live prompt abuse in AI agents. |
| A5 — Excessive Agency | Directly relates to overbroad action authority that testing may miss and runtime must constrain. | |
| Recommendation — Test and monitor agent prompts for injection resistance across both evaluation and live use. Limit agent authority and validate that runtime checks block unsafe actions. | ||
| NIST AI RMF | GOV — Govern | Sets lifecycle AI risk governance that spans assessment and operational oversight. |
| MAP — Map | Requires identifying context, use, and impact before deployment and during operation. | |
| MEASURE — Measure | Supports testing and ongoing measurement of AI system behaviour and risk signals. | |
| Recommendation — Define accountability for both pre-deployment assurance and runtime AI risk oversight. Map agent use cases, dependencies, and impact before release and as conditions change. Measure agent behaviour, control effectiveness, and drift with repeatable evaluation criteria. | ||
| MITRE ATLAS | ATK-000 — AI System Exploitation | Covers adversarial behaviours that exploit AI systems during testing and runtime. |
| Recommendation — Model adversarial abuse paths and use them to harden both testing scenarios and runtime detection. | ||
| CSA MAESTRO | THR-01 — Threat Modeling | Applies to threat modeling agent workflows before release and under live abuse paths. |
| Recommendation — Threat-model agent workflows to separate design weaknesses from live exploitation paths. | ||
Practitioner Guidance
What to prioritise: Treat release testing as a qualification gate and runtime security as a containment layer. If an agent can read, write, or trigger actions outside its own boundary, the runtime controls deserve the same attention as the evaluation plan.
What to verify: Confirm that the test environment includes realistic prompts, tool paths, and data conditions, then verify that runtime logs, approvals, and policy checks are actually able to stop or reverse unsafe actions. The control is weak if it only detects abuse after the effect has already occurred.
Common mistake: Teams often rely on benchmark-style testing and assume the result generalises to production. That is risky when the live system can encounter different users, untrusted content, or changing retrieval sources.
Practitioner takeaway: The right model is not “test first or monitor later” but “test to expose weakness, then monitor to contain what testing cannot prove.”
Related resources from NHI Mgmt Group
- How do security teams balance pre-deployment testing and runtime validation for AI systems?
- What is the difference between SAST tools and runtime security tools for AI coding agents?
- What is the difference between AI code reasoning and runtime security testing?
- How should security teams govern cloud AI agents at runtime instead of relying only on pre-deployment reviews?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 7, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
