AI jailbreaks and foundation model safeguards: where do they fail?

TL;DR: AI jailbreaks use creative prompts, roleplay, and other coercive techniques to bypass model safeguards and extract restricted output from systems such as ChatGPT, Claude, and Gemini, according to ZioSec. The practical lesson is that prompt filters alone do not close the governance gap when sensitive information can be elicited through indirect, adversarial instruction.

NHIMG editorial — based on content published by ZioSec: Exploring AI Jailbreaks: Techniques and Risks in Foundation Models

Questions worth separating out

Q: How should security teams reduce the risk of AI jailbreaks in model-enabled workflows?

A: Security teams should treat jailbreak risk as a control-design problem, not only a prompt-filter problem.

Q: Why do AI jailbreaks matter for identity and access governance?

A: AI jailbreaks matter because models increasingly sit inside access paths to data and tools.

Q: What do security teams get wrong about model safety filters?

A: Teams often assume safety filters stop harmful output in the same way an access control stops unauthorised access.

Practitioner guidance

• Classify model context as sensitive data exposure surface Map which prompts, system instructions, and retrieved documents can reveal confidential information if a jailbreak succeeds.
• Separate generation from execution authority Do not let model output directly trigger privileged actions, tool calls, or data exports.
• Red-team indirect prompt paths, not only obvious abuse Test poetry, roleplay, fictional narration, translation, and multi-turn steering techniques against any model that handles internal content.

What's in the full article

ZioSec's full post covers the prompt patterns and jailbreak examples this analysis intentionally leaves at the conceptual level:

• The exact corporate-horror and poetry-style prompt constructions used to coerce the model into revealing hidden information
• The step-by-step evolution of the jailbreak that exposed system prompt content during iterative testing
• The article's own examples of how subtle prompt framing can surface confidential details from a chatbot
• The researchers' discussion of reporting the findings and limiting disclosure to reduce reuse across models

👉 Read ZioSec's analysis of AI jailbreak techniques and foundation model risks →

AI jailbreaks and foundation model safeguards: where do they fail?

Explore further

View Full Forum →  |  NHI Foundation Course →