Adaptive defenses are now central to LLM security and utility

At a glance

What this is: Lakera’s research argues that LLM security cannot rely on fixed prompt-time controls because attackers adapt and user utility still has to be preserved.

Why it matters: IAM and security teams need this lens because LLM governance now depends on controlling what an AI system can do over time, not just what it is asked to do in a single interaction.

Context

Adaptive LLM security is the problem of keeping a model safe as attacker behaviour changes during interaction, while still leaving the application useful for legitimate users. That puts the governance question squarely on how controls behave over the full session, not just at the first prompt.

The article frames this as a trade-off between security and utility, especially where the system prompt is doing too much of the control work. For teams responsible for LLM-enabled applications, the practical gap is that a block list or static policy can look effective while quietly degrading the user experience or missing adaptive attackers.

The research is primarily about LLM application governance and agentic behaviour boundaries, not generic model performance. The starting position is typical for teams moving from experimentation to production: a simple control surface becomes inadequate once users and attackers both begin to probe it.

Key questions

Q: What breaks when LLM defenses stay static while attackers adapt?

A: Static defenses fail because the attacker learns from each model response and adjusts the next prompt until the control is bypassed. The result is a false sense of safety: the first attempts may be blocked, but the policy becomes weaker as the session continues. Teams should test defenses against iterative probing, not only single-shot attacks.

Q: When should security teams prioritise utility over tighter LLM blocking?

A: They should never choose utility instead of security, but they must evaluate both together whenever a tighter block begins to disrupt legitimate tasks. If a defense reduces answer quality or refuses benign requests, the application may no longer be fit for purpose. The right balance depends on whether the workflow is informational, operational, or high-risk.

Q: How should security teams evaluate LLM defenses in production?

A: They should evaluate both attack resistance and user utility. A defense that blocks more malicious prompts but also degrades legitimate completion quality may be operationally unacceptable. Good evaluation uses realistic sessions, measures attacker adaptation over time, and compares security gains against the loss in usability before deciding what to deploy.

Q: What is the difference between prompt filtering and layered LLM defense?

A: Prompt filtering is a single enforcement point that tries to stop unsafe requests at the input stage. Layered defense combines multiple controls, such as scope restriction, session monitoring, and downstream policy enforcement, so one bypass does not expose the whole application. In practice, layered controls are more resilient against attackers who learn from feedback.

Technical breakdown

Why static prompt defenses fail under adaptive attackers

A static defense assumes the attacker’s strategy is fixed enough that one policy decision can remain effective. In LLM applications, that assumption breaks because attackers get feedback from each interaction and can refine prompts, sequencing, and wording until the defense yields. The article’s D-SEC framing captures this by treating attacker learning as part of the threat model rather than noise. That makes security a moving target, not a single enforcement point. Practical implication: evaluate controls as session behaviours, not isolated prompt rejections.

Practical implication: Measure defenses against adaptive attacker behaviour across sessions, not only against first-attempt blocking.

Security-utility trade-offs in LLM application design

LLM security is not equivalent to refusing more requests. A control that blocks too aggressively can shorten answers, reduce response quality, or deny benign users what they need, which is why utility must be measured alongside protection. The article highlights this especially for system-prompt-based controls, where stronger restrictions can change the character of the application itself. That means the security boundary is also a product boundary. Practical implication: treat usability loss as a control failure mode, not just a user-experience issue.

Practical implication: Test whether stronger controls still preserve the application’s intended usefulness for legitimate users.

Defense in depth for LLMs and autonomous agents

The article argues that different defenses specialise in different attack patterns, so combining them raises the attacker’s cost more effectively than relying on one mechanism. That is especially relevant where the system behaves more like an autonomous workflow than a simple chatbot, because the control has to work across multiple decision points and interactions. Narrowing the application scope also reduces attack surface by limiting what the model is allowed to do in the first place. Practical implication: build layered controls that constrain scope, monitor interaction history, and limit the attacker’s budget.

Practical implication: Use layered enforcement so one bypass does not expose the full application path.

Threat narrative

Attacker objective: The attacker’s objective is to bypass static LLM safeguards by learning from feedback until the model reveals restricted content or behaviour.

1. Entry begins with the attacker interacting with the model through normal prompts and probing how the system responds to borderline requests.
2. Credential or control access is not the issue here; the attacker instead learns the defense logic from model feedback and uses that feedback to refine the next attempt.
3. Escalation occurs as repeated interaction reveals which prompt shapes, sequencing, or framing bypass the static safeguard and reach disallowed outputs.
4. Impact is the disclosure of sensitive or restricted model behaviour while the application remains vulnerable to the attacker’s iterative adaptation.

Breaches seen in the wild

• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Static LLM defenses create security theatre once attackers can adapt in-session. A control that works against the first prompt may fail after the model reveals how it is enforcing policy. That is not a minor tuning issue, it is a governance failure in the control model itself. Practitioners should treat attacker learning as a first-class design constraint, not an edge case.

Security-utility trade-off is now the real control surface for LLM governance. If a defense keeps attacks out but degrades legitimate output quality, the application has still failed its purpose. That is why red-teaming has to measure both resistance and usefulness, especially where system prompts carry too much policy burden. Teams need to govern the service boundary, not just the prompt.

D-SEC is a useful named concept because it reframes LLM risk as a dynamic control problem. The model recognises that attackers, users, and developers all shape the session outcome, which is closer to production reality than one-time benchmark testing. The implication is that LLM security programmes need session-aware testing and policy design, not static prompt hardening alone.

Restricting application scope is often more defensible than widening the defense stack indefinitely. If an LLM only needs to handle a narrow class of tasks, governance should make that boundary explicit and enforceable. Wider scope expands the attack surface and raises the utility penalty for every control. Practitioners should start by asking what the model should never be allowed to do.

Autonomous behaviour magnifies the cost of brittle defenses. When an LLM influences a workflow with its own branching decisions, a weak control does not just leak one answer, it can steer follow-on actions. That makes layered defenses and interaction-history analysis more important as the system becomes more agent-like. Security teams should design for repeated decision points, not single-shot prompts.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: Agentic AI Security Guide

What this signals

Adaptive control is now the governance baseline for LLM applications. Static prompt rules may still filter obvious abuse, but they do not hold up when the attacker can learn from each response and reshape the next attempt. Security teams should assume the control itself will be probed and adaptively tested in production.

Security-utility trade-offs will increasingly decide whether an LLM is viable at all. If the strongest control materially degrades legitimate work, the programme has only shifted risk from abuse to adoption failure. That is why the control conversation has to include response quality, refusal behaviour, and session continuity, not just attack blocking.

According to the State of Secrets Sprawl 2026, AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers. That pattern reinforces the article’s core point: the wider AI control plane can become the weakest link even when the model itself is tightly governed.

For practitioners

• Measure security and utility together Test each defense against both malicious prompts and benign tasks so you can see where protection starts damaging response quality or task completion.
• Restrict application scope explicitly Define the narrowest task boundary the model actually needs, then encode that boundary in system instructions and policy checks so unsupported requests fail closed.
• Layer distinct defenses Combine prompt controls, session-level review, and downstream policy enforcement so a bypass in one layer does not expose the full application path.
• Use session history to detect adaptation Track repeated suspicious interactions over the life of the session and cut off probing that shows the attacker is learning the control logic.
• Red-team for utility loss as well as bypass Score each test not only on whether it blocks the attack but also on whether legitimate use cases still work at acceptable quality.

Key takeaways

• Static LLM defenses do not remain reliable once attackers can learn from model feedback over a session.
• The security problem is no longer just blocking abuse, but doing so without breaking legitimate usability.
• Layered controls and narrower scope are more defensible than relying on a single prompt-time safeguard.

Key terms

• Adaptive LLM Defense: An adaptive LLM defense changes behaviour as attacker interaction changes, rather than relying on one fixed prompt rule or filter. In practice, the control uses session signals, repeated probing patterns, and layered enforcement to keep pace with adversarial learning without making the application unusable for legitimate users.
• Security-Utility Trade-off: The balance between preventing malicious behaviour and preserving enough legitimate function for users to complete their tasks. In AI systems, stricter controls can reduce risk but also shorten responses, block valid requests, or limit usefulness, so the acceptable balance must be set explicitly.
• Defense in depth: Defense in depth is the practice of stacking independent controls so one failed check does not expose the whole system. In App Router authentication, that means verifying identity in middleware, route handlers, and data access logic, because each layer protects a different part of the request path.
• Session Completion Rate: Session Completion Rate measures how often a user or attacker finishes a session successfully under a given defense. In this article’s context, it is useful because it shows whether a safeguard blocks malicious paths without over-blocking legitimate use across the interaction lifecycle.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.