Topic starter
18/08/2026 2:50 pm
TL;DR: LLM alignment is often shallow, because refusal behavior can collapse once harmful generation is already underway, according to VirtueAI research, but a small inference-time method called Any-Depth Alignment restores refusals mid-stream and cuts prominent prompt-attack success below 3%. The finding matters because safety controls for LLMs and agentic systems must work during generation, not only at turn start.
Editorial analysis by NHI Mgmt Group, based on content published by VirtueAI: “Virtue AI Research”.
By the numbers:
- ADA reduces the average success rate of prominent adversarial prompt attacks to below 3%.
Key questions
Q: What breaks when LLM refusal only works at the start of a response?
A: The model can remain unsafe once a harmful continuation is underway, because the original refusal state may not survive deeper into generation.
Q: Why do harmful prefixes and assistant-prefill attacks increase LLM risk?
A: They place the model into a generation trajectory where the initial refusal boundary is already bypassed.
Q: How can security teams tell whether an LLM is actually safe during generation?
A: They should test refusal durability at multiple depths, including long harmful prefixes, mid-response interruptions, and resumption cases.
Practitioner guidance
- Map safety controls to generation depth Test whether refusal holds after harmful continuations begin, not only at prompt entry, and record where the model loses safety state.
- Add mid-stream policy resets to inference pipelines Insert a control point that can reassert safety instructions or headers during long generations, especially for agentic and tool-using systems.
- Evaluate prompt-attack resistance under adversarial prefill Benchmark long harmful prefixes against the models you actually deploy, including cases where the attack spans dozens or thousands of tokens.
Bottom line: LLM safety can degrade after generation begins, which leaves a gap between initial refusal and runtime behavior.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
18/08/2026 3:32 pm
Any-depth safety is now a governance requirement, not a model nicety. The paper shows that refusal quality is depth-dependent, which means safety cannot be assessed only at the first response token or prompt ingress. For AI governance teams, that changes the unit of control from prompt screening to runtime assurance, especially in systems that chain LLM output into tools or policies. The practitioner conclusion is simple: safety must persist for the full generation path.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should organisations treat LLM safety as a model issue or a runtime governance issue?
A: Both, but runtime governance is the decisive layer when the model is already in production. A model may be aligned in isolation and still fail once generation begins inside an agentic workflow. That is why policy enforcement, monitoring, and intervention points have to sit inside the execution path, not only around the prompt.
👉 Read our full editorial: Any-depth alignment exposes a new failure mode in LLM safety