A bypass technique that uses images, documents, embedded text, or mixed-input prompts to evade a model’s safety rules. It matters because controls built for text-only systems often do not hold when the model can process multiple input types at once.
How multimodal jailbreaks work
Multimodal jailbreaks exploit the fact that safety controls are often stronger, and more extensively tested, on plain text than on images, documents, screenshots, OCR output, embedded captions, or mixed prompt inputs. Attackers use that gap to hide malicious instructions, split a harmful request across modalities, or place conflicting cues in one input so the model follows the wrong path.
The core idea is not a new model weakness, but a change in the input path. A system that sanitises text prompts well can still be vulnerable if it gives the model broad freedom to interpret visual or document content without equivalent policy enforcement. That is why jailbreaks in this category are often described as model security failures rather than simple prompt-typing tricks.
Why multimodal inputs expand the attack surface
Multimodal systems increase the number of places where an attacker can hide intent. A malicious instruction may appear as an image overlay, a scanned document annotation, a table, a chart label, or text embedded in a file the user believes is harmless. If the model fuses those signals into a single reasoning stream, the attacker gains more opportunities to confuse alignment filters, policy classifiers, or human reviewers.
This matters because the model may treat content as data in one stage and as instructions in another. A screenshot can contain text that is visible to the model but not obvious to the operator, and a document can combine narrative, formatting, and metadata in ways that make policy enforcement inconsistent. For deeper context on the safety and security distinction that shapes how teams govern these systems, see AI Safety vs AI Security Guide.
Detection and evaluation challenges
Testing multimodal jailbreak resistance requires more than running a text prompt suite. Organisations need to evaluate how the system behaves when instruction-like text appears inside images, PDFs, office files, transcripts, or mixed media, and whether the model exposes those hidden instructions in its intermediate interpretation. Gaps often show up where the pipeline OCRs, summarises, translates, or re-encodes the content before policy checks are applied.
A useful review process looks for whether the model can be induced to ignore safety rules, reveal restricted content, or follow a malicious secondary instruction that arrived through a non-text channel. That is why red teaming AI agents for identity abuse is relevant even beyond agent-specific scenarios, because jailbreak-style testing often overlaps with instruction hijacking, approval bypass, and unintended tool use.
Controls and defensive design for multimodal systems
Defence starts with treating every modality as potentially instruction-bearing, not just plain text. Teams should apply policy checks after ingestion, after OCR or parsing, and again before the model acts on extracted content. They also need input normalisation, content provenance checks, separation between user data and system instructions, and clear rules for when visual or document content may influence downstream actions.
Models are safer when multimodal content is constrained by the same governance expectations as text, rather than being routed around them. That means reviewing how the system ingests files, how it labels extracted text, and how it distinguishes user-provided material from embedded instructions. For a broader threat-model view, AI Threats and Model Security Guide is the most direct companion reference, because multimodal jailbreaks sit inside the larger class of model manipulation and extraction risks.
Risk and Threat Considerations
Multimodal jailbreaks are risky because they exploit trust in input channels that defenders often treat as passive data. When the model can read images or documents as if they were instructions, adversaries can bypass text-only filters, smuggle policy-breaking content past reviewers, or trigger unsafe tool use in downstream workflows.
Failure mechanism: The attacker places malicious instructions inside a non-text modality, or splits the payload across modalities, so the model fuses harmful content into its reasoning before safety controls can reliably classify it.
Impact: The model may reveal restricted information, ignore policy boundaries, generate unsafe content, or carry attacker intent into connected systems, especially where multimodal output feeds action, summarisation, or approval workflows.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST AI RMF and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | T1204 — User Execution | Multimodal jailbreaks rely on crafted content to influence user or model-driven execution. |
| Recommendation — Map malicious multimodal content to T1204 and validate where hidden instructions can alter execution. | ||
| NIST AI RMF | GV.1 — Govern | AI governance must account for multimodal prompt and content abuse risks. |
| Recommendation — Govern multimodal input handling as part of AI risk management and accountability. | ||
| NIST SP 800-53 Rev 5 | SI-10 — Information Input Validation | Multimodal jailbreaks exploit weak validation of non-text and parsed inputs. |
| Recommendation — Apply SI-10 checks to validate OCR, parsed documents, and other model inputs before use. | ||
Practitioner Guidance
What to watch for: The biggest mistake is assuming that text safety testing covers all user inputs. Practitioners should review OCR, document parsing, screenshot handling, and any feature that turns visual content into model-readable text, because that is where hidden instructions often become operationally significant.
Practitioner takeaway: If a system can read it, the system can be influenced by it, so multimodal safety testing should cover every input path the model can interpret.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on November 4, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org