The model can remain unsafe once a harmful continuation is underway, because the original refusal state may not survive deeper into generation. That creates a control gap between prompt-time filtering and runtime safety. For agentic systems, the danger is bigger because unsafe output can be handed directly to tools, memory, or downstream automation.
When the refusal only exists at the prompt boundary, what is still allowed to happen?
A front-end refusal is only effective if it persists through the full generation path. If the model can begin a harmful completion and then lose the refusal constraint later, the safety control has become a one-shot filter rather than a runtime guard. That matters most when downstream tooling can act on partial output before the model fully “changes its mind.”
The technical failure is not just that unsafe text may appear, but that the model may continue generating a harmful trajectory after the initial objection. In practice, this creates a split between policy enforcement at the start and behavior during token-by-token decoding, which is where agentic systems, streaming interfaces, and tool-using workflows can turn a partial unsafe response into action.
For that reason, refusal should be treated as a property of the whole decode path, not a greeting at the top of the answer. A system that only checks the first token or first sentence can still leak actionable detail, cite harmful steps, or produce output that downstream components interpret as permission to proceed.
Why does this become more dangerous in agentic workflows?
In agentic systems, the model output is not just read by a person. It may be parsed, routed, summarized, or passed directly into tools, memory, or orchestration logic. If the refusal state is fragile, a later continuation can convert an initially blocked response into an unsafe instruction stream, and that is enough to create a control gap between policy intent and machine execution.
That is why runtime safety has to be evaluated alongside prompt safety. A guard that works only at the beginning can still fail when the agent is asked to elaborate, translate, or “just complete the thought,” because those are exactly the moments when hidden unsafe continuation can reappear.
When the model is embedded in a workflow, the risk is amplified by partial trust. A human may see the refusal, but a tool may consume the continuation; or the reverse may happen, where the unsafe continuation is invisible in a UI but still present in an internal channel. The relevant question is not whether the model refused once, but whether the system can keep that refusal binding until generation stops.
What control boundary needs to move from prompt-time to runtime?
The core boundary should shift from input screening to output supervision. If safety only exists at the prompt, then the model is vulnerable to refusal drift, streaming leakage, and policy bypass through continuation. If safety is enforced during decoding, post-processing, and tool handoff, then the system can stop unsafe material before it becomes operationally useful.
This is also where internal links to broader AI security practice become useful. A layered agent threat model is captured well in Agentic AI Security Guide, especially where the answer depends on whether outputs can reach tools and orchestration steps. For memory handling, AI Agent Memory Security Guide shows why unsafe continuations become worse when they are retained or reused across turns. And where the problem is supply-chain or key exposure rather than generation safety alone, LLM Provider API Key Security and LLMjacking Guide helps frame the blast radius of downstream abuse.
Practitioners should think in terms of enforcement points: prompt filter, decoder, stream processor, tool router, and memory write path. A refusal that is not enforced at each relevant boundary can be bypassed by continuation, transformation, or handoff.
Risk and Threat Considerations
When refusal only works at the start of generation, the main risk is control leakage across the rest of the response. That can expose unsafe instructions, enable prompt-injection style steering, or let an agent hand harmful content to tools before a later safeguard intervenes. The weakness is especially material in streaming and automation-heavy systems where output is consumed incrementally.
Failure mechanism: The model passes an initial refusal check, then decodes into an unsafe continuation because the safety state is not persisted or re-evaluated during generation and handoff.
Impact: Harmful content can reach the user, a downstream model, memory, or a tool invocation path, which turns a cosmetic refusal into a real execution-risk gap.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack and risk surface, while NIST AI RMF and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | LLM refusal drift matters when agent output can trigger tool or privilege use. |
| Recommendation — Enforce runtime authorization so unsafe generations cannot become tool actions. | ||
| NIST AI RMF | GOVERN — GOVERN | The issue is AI safety governance over when a model response remains constrained. |
| Recommendation — Define and test safety controls across the full generation lifecycle. | ||
| NIST SP 800-53 Rev 5 | SI-10 — Information Input Validation | Prompt and output filtering are validation controls that must hold beyond the first token. |
| AC-3 — Access Enforcement | Unsafe continuations become dangerous when they can reach tools or downstream automation. | |
| Recommendation — Validate model outputs continuously, not only at prompt entry. Enforce access decisions at every tool and automation boundary. | ||
Practitioner Guidance
What to verify: Test refusal persistence across the full generation lifecycle, not just at the first token. Include streaming output, regeneration, summarization, and tool-call boundaries in the test plan, because that is where the control usually fails.
Decision rule: If unsafe content can be produced after the initial refusal and before final output is blocked, treat the system as inadequately protected even if the opening sentence looks compliant. A good control should fail closed at every downstream boundary that can execute, store, or relay the content.
Practitioner takeaway: A refusal that does not survive decoding is not a safety control, it is only a delay before unsafe output reaches a more powerful part of the system.