Any-depth alignment in LLMs: can refusal hold mid-generation?

TL;DR: LLM alignment is often shallow, because refusal behavior can collapse once harmful generation is already underway, according to VirtueAI research, but a small inference-time method called Any-Depth Alignment restores refusals mid-stream and cuts prominent prompt-attack success below 3%. The finding matters because safety controls for LLMs and agentic systems must work during generation, not only at turn start.

NHIMG editorial — based on content published by VirtueAI: Any-Depth Alignment, Unlocking Innate Safety Alignment of LLMs to Any-Depth

By the numbers:

• ADA secures a near-100% refusal rate against adversarial prefill attacks ranging from dozens to thousands of tokens.
• ADA reduces the average success rate of prominent adversarial prompt attacks to below 3%.

Questions worth separating out

Q: How should security teams test whether LLM safety controls still work after harmful generation starts?

A: Test across long contexts, adversarial continuation, and assistant-prefill scenarios, not just first-turn prompts.

Q: Why do shallow refusal controls fail in long LLM interactions?

A: They often depend on prompt position and sequence context, so the model’s refusal tendency is strongest at the start of a response and weaker after harmful continuation begins.

Q: What do security teams get wrong about LLM safety at runtime?

A: They assume a single moderation step at prompt ingress is enough.

Practitioner guidance

• Test refusal behavior at multiple generation depths Evaluate model responses against adversarial continuation, long prefill, and multi-turn escalation so you can see where refusal breaks after the initial token boundary.
• Add runtime re-evaluation before tool use Require the assistant to re-check harmfulness before it emits tool instructions, code, or delegated tasks.
• Instrument generation-depth telemetry Log where refusals occur, where harmful continuations begin, and whether re-prompting changes model behavior.

What's in the full report

VirtueAI's full research covers the mechanism detail this post intentionally leaves for the source:

• Empirical attack testing across adversarial prefill lengths from dozens to thousands of tokens.
• Comparative refusal-rate results across open-source model families including Llama, Gemma, Mistral, Qwen, DeepSeek, and gpt-oss.
• The specific inference-time token insertion approach used to re-trigger refusal behavior.
• Benchmark notes on how the method performs after subsequent instruction tuning.

👉 Read VirtueAI's paper on Any-Depth Alignment for LLM safety →

Any-depth alignment in LLMs: can refusal hold mid-generation?

Explore further

View Full Forum →  |  NHI Foundation Course →