Multimodal Jailbreak

A bypass technique that uses images, documents, embedded text, or mixed-input prompts to evade a model’s safety rules. It matters because controls built for text-only systems often do not hold when the model can process multiple input types at once.

Expanded Definition

Multimodal jailbreak refers to an attempt to defeat model safeguards by combining text with other input channels such as images, scanned documents, screenshots, audio transcripts, or file attachments. In practice, the bypass works because the harmful instruction is hidden, fragmented, or encoded in a format that the model processes differently from plain text. For NHI Management Group, the key point is that the attack is not limited to prompt wording alone. It exploits the full input surface of an AI system, especially where a large language model, retrieval layer, OCR pipeline, or vision component is chained together. Definitions vary across vendors because some treat it as a prompt-injection subtype, while others classify it as a broader multimodal safety failure. The most useful operational view is that any trust boundary between modalities can become an avenue for policy evasion. Standards are still evolving, but NIST Cybersecurity Framework 2.0 remains relevant because it frames how organisations should govern protective controls across the system lifecycle. The most common misapplication is assuming text-only red teaming is sufficient, which occurs when organisations test chat prompts but ignore uploaded images, PDFs, and tool-fed content.

Examples and Use Cases

Implementing multimodal safety rigorously often introduces workflow friction, because organisations must inspect and filter more than one content type while preserving legitimate user productivity.

• A user uploads a screenshot containing hidden or stylised instructions that the model reads through OCR, causing the safety layer to miss the actual intent.
• A malicious PDF includes prompt-like text in headers, footers, or annotations, and the model follows it when summarising or extracting data.
• An image contains embedded instructions or steganographic cues that influence a vision-language model during captioning, classification, or reasoning.
• A document plus follow-up chat message creates a compound attack where the unsafe instruction appears harmless until the model combines both inputs.
• A retrieval-augmented workflow pulls in manipulated content from a knowledge source, and the model treats the retrieved artefact as trusted context instead of untrusted input.

For structured model testing, teams often borrow the defensive mindset used in OWASP guidance for large language model applications, even though multimodal jailbreaks are still an evolving category rather than a single fixed control class. The practical lesson is to test every supported input path, not just the chat box.

Why It Matters for Security Teams

Multimodal jailbreaks matter because they expose a gap between policy and implementation. Security teams may have strong text filtering, yet still allow unsafe instructions through image parsing, document ingestion, or agent tool pipelines. That creates direct risk for data leakage, policy bypass, unsafe tool execution, and manipulation of downstream workflows that depend on AI outputs. The issue becomes more serious when AI systems are connected to NHI, because service accounts, API keys, and automated agents may act on corrupted model outputs without human review. In that setting, the attack is not just about content moderation. It becomes an identity and authorisation problem as well, because the model may be induced to request, reveal, or route secrets in ways that bypass normal controls. Security programmes should align detection, content sanitisation, permission boundaries, and human approval steps across every modality the model accepts. OWASP’s LLM risk guidance is useful for framing these failures as application-security issues, while governance models such as NIST Cybersecurity Framework 2.0 help assign ownership and response responsibilities. Organisations typically encounter the full impact only after a seemingly benign upload triggers unsafe output or an automated workflow executes a bad instruction, at which point multimodal jailbreak becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• How should security teams use root and jailbreak detection in mobile banking?
• Why do multimodal video platforms create new IAM and audit risks?
• How should security teams defend enterprise AI systems against jailbreak attacks?
• How can organisations reduce jailbreak risk without slowing AI adoption?