Topic starter
05/12/2025 4:58 pm
TL;DR: Adversarial AI explainability can be used to study LLM internals and craft new jailbreak variants that bypass safeguards in both open- and closed-source models, according to CyberArk, while also stressing that LLM output cannot be treated as a security control. That turns jailbreak resistance into an application-governance problem, not just a model-safety problem.
Editorial analysis by NHI Mgmt Group, based on content published by CyberArk: “Unlocking New Jailbreaks with AI Explainability”.
Key questions
Q: How should security teams layer defenses around LLM applications when jailbreaks keep succeeding?
A: Treat model alignment as one control, not the control.
Q: Why do jailbreaks become a serious security risk when AI agents have tool access?
A: Jailbreaks matter more in tool-enabled environments because the model is no longer just generating unsafe text.
Q: What signs show that a workflow is trusting LLM output too much?
A: Warning signs include prompts that can trigger downstream actions, responses that are executed without review, and controls that assume the model will always refuse unsafe requests.
Practitioner guidance
- Separate model output from enforcement logic Require a non-LLM control layer to decide whether a prompt, response or inferred action is allowed to proceed.
- Instrument jailbreak-style prompt testing Test harmful prompts, encoding transformations and indirect prompt variants against your own workflows to see where refusal behaviour breaks and where downstream systems still trust the output.
- Constrain agentic workflows with explicit approval gates If an LLM can influence routing, execution or permissions, require deterministic gates outside the model before any action is taken.
Bottom line: AI explainability can help reveal why some jailbreaks work, but the same insight can also make attacks more effective if it is not governed.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 10 months ago by Abdelrahman
This topic was modified 4 days ago by NHI Mgmt Group
08/10/2026 5:41 pm
LLM output cannot be treated as a security control: CyberArk's research reinforces a basic governance boundary that many AI programmes still blur. A model may assist with policy decisions, but it cannot safely act as the policy enforcement point. The implication is that approval, validation and access decisions must live outside the model, or jailbreaks become control-plane failures.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should organisations use explainability findings to improve model safety?
A: Yes, but only as one input to a layered control design. Explainability can help identify weak layers or refusal patterns, yet it should inform validation, gating and monitoring rather than becoming the trust basis for the model itself.
👉 Read our full editorial: AI explainability is creating new jailbreak paths for LLM security