Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

LLM jailbreaking: are your guardrails keeping up with attacks?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 9439
Topic starter  

TL;DR: LLM jailbreaks use roleplay, prompt injection, prompt leaking, and other adversarial inputs to push models past safety rules and into harmful or policy-violating output, according to Lakera. The security problem is not just prompt hygiene but the assumption that model behaviour stays bounded once access is granted.

NHIMG editorial — based on content published by Lakera: Jailbreaking Large Language Models: Techniques, Examples, Prevention Methods

By the numbers:

  • 502.249 tokens, pts average 502.249 tokens, compared with 178.686 tokens for regular prompts.
  • Jailbreak prompts have a toxicity score of 0.150, compared with 0.066 for regular prompts.
  • OWASP's Top 10 for LLM contains 10 security and safety issues that developers and security teams must consider when building LLM applications.

Questions worth separating out

Q: How should security teams stop jailbreak prompts from becoming unsafe actions?

A: Security teams should treat jailbreak resistance as a control-chain problem.

Q: Why do LLM jailbreaks remain effective even when models have safety filters?

A: Jailbreaks remain effective because the model is still optimized to follow context, and safety rules are probabilistic rather than absolute.

Q: What do security teams get wrong about prompt injection risk?

A: The common mistake is treating prompt injection as a content-moderation issue instead of a trust-boundary issue.

Practitioner guidance

  • Separate untrusted input from system instructions Keep user content, retrieved content, and developer instructions in distinct processing layers so the model cannot treat them as equivalent.
  • Limit what the model can do after a jailbreak Restrict tool access, data retrieval, and action execution so a compromised prompt cannot become an unauthorised workflow.
  • Red team prompts that look normal Test with long, roleplay-heavy, and semantically plausible prompts rather than only obvious toxic inputs.

What's in the full article

Lakera's full blog covers the operational detail this post intentionally leaves for the source:

  • Prompt-by-prompt examples of universal jailbreak techniques and how they differ in structure.
  • Detailed breakdown of prompt injection, prompt leaking, DAN, roleplay jailbreaks, and token-system attacks.
  • Prevention methods including red teaming, contextual analysis, and automated stress testing.
  • Examples and research references showing how adversarial prompts can influence model output.

👉 Read Lakera's guide to LLM jailbreaking techniques, examples, and prevention →

LLM jailbreaking: are your guardrails keeping up with attacks?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 8923
 

Jailbreaking exposes an instruction-trust failure, not just a content-safety problem. The article shows that models can be steered by adversarial context even when the original task appears constrained. That means the core governance assumption, that a model will reliably privilege the developer's intent over attacker-supplied text, is already broken in practice. Practitioners should treat prompt trust as a control boundary, not a content filter.

A few things that frame the scale:

  • 71% of NHIs are not rotated within recommended time frames, increasing the risk of compromise over time, according to Ultimate Guide to NHIs.
  • Only 5.7% of organisations have full visibility into their service accounts, which shows how often identity governance starts without complete inventory, according to Ultimate Guide to NHIs.

A question worth separating out:

Q: Who is accountable when a jailbroken model causes an unsafe enterprise action?

A: Accountability sits with the organisation that allowed model output to become operational without sufficient controls. Governance should assign ownership for prompt handling, tool permissions, output validation, monitoring, and incident response. If an LLM can act, the identity and authorisation model must define who can approve that action.

👉 Read our full editorial: LLM jailbreaking shows why prompt controls alone are not enough



   
ReplyQuote
Share: