Cross-modal prompt injection is an attack that hides malicious instructions in one data type, such as text, image, audio, or video, so an AI agent follows them during processing. In security terms, it exploits model interpretation across modalities to override intended controls, trigger unsafe actions, or leak sensitive data.
What Cross-Modal Prompt Injection Actually Exploits
Cross-modal prompt injection works because modern AI systems do not process every input as a cleanly separated signal. Text, images, audio, video, and embedded metadata can all carry instructions that a model may interpret as higher-priority guidance unless the system is explicitly designed to distinguish content from commands.
This makes the attack broader than ordinary prompt injection. The malicious instruction can be hidden in a chart image, a spoken phrase in audio, overlaid text in video, or a document that mixes human-facing information with machine-readable cues. The security problem is not the medium itself, but the model’s willingness to treat the medium as instruction-bearing.
Why Cross-Modal Prompt Injection Is Dangerous
The danger is that one compromised modality can influence the whole interaction. A user may believe they are supplying harmless reference material, while the agent silently follows attacker-supplied directions that were embedded inside that material.
That can override intended guardrails, redirect tool use, expose private context, or cause the system to perform actions that the user never asked for. In agentic systems, the issue is amplified because the model may not just answer differently, it may act differently.
For a broader view of how these attacks are framed, NHIMG’s Prompt Injection Guide covers direct, indirect and multimodal attacks, along with common exfiltration paths and layered defenses.
Where Organisations Usually Miss It
Cross-modal injection often enters through ordinary business content, not obviously malicious payloads. Examples include uploaded screenshots, meeting recordings, customer attachments, training videos, product images, or documents that combine visible text with hidden instructions.
The exposure grows when models are allowed to consume untrusted inputs alongside privileged context, connected tools, or retrieval systems. A system that can read, summarise, search, or act on what it receives must be able to separate user intent from adversarial instruction across every modality it supports.
Security teams should treat multimodal ingestion paths, file parsers, OCR, speech-to-text, and vision pipelines as part of the attack surface, not as neutral preprocessing steps.
How Defenders Reduce the Attack Surface
The most effective defences are architectural, not cosmetic. Systems need content isolation, trust boundaries between sources, instruction hierarchy enforcement, and tool-use controls that do not blindly inherit instructions from any input channel.
Practical hardening also includes limiting what untrusted modalities can influence, filtering or sanitising risky content before it reaches the model, and constraining the model’s ability to act on instructions extracted from unverified sources. Where agents can call tools, the model should not be able to convert interpreted content into high-impact action without an explicit policy check.
For agent-focused control design, the Agentic AI Security Guide is a useful internal reference for controls around inputs, memory, tools, orchestration and identity, while the OWASP Agentic AI Top 10 provides an external framework for the broader risk landscape.
How to Detect and Investigate It
Detection usually requires looking for inconsistency between the user’s stated intent and the model’s actual behavior. A suspicious sign is when a model follows instructions that were not present in the conversation but were embedded in an uploaded or observed artifact.
Investigation should preserve the original artefact, the modality-specific extraction path, and the agent trace, including OCR results, transcribed audio, frame captures, and tool calls if the model acted on the injection. That evidence helps determine whether the issue was prompt parsing, retrieval contamination, tool misuse, or a failure of instruction hierarchy.
If the system handled browser-based or desktop-based workflows, Browser and Computer-Use Agent Security Guide is relevant because session-scoped agents can turn a hidden instruction into real-world action through the user’s authenticated context.
Risk and Threat Considerations
Cross-modal prompt injection is risky because it turns ordinary content channels into covert command paths. When the model can interpret untrusted text, images, audio or video as instructions, an attacker may be able to steer the system into leaking data, misusing tools, or bypassing the user’s intended workflow.
Failure mechanism: The attacker hides directive language inside a modality the system trusts for comprehension, then relies on the model to merge that hidden instruction with legitimate context and act on it.
Impact: The result can be secret exposure, unauthorized actions, corrupted outputs, or agent behavior that appears internally consistent but is externally attacker-controlled.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack and risk surface, while NIST AI RMF sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI01 — Agent Goal Hijack | Cross-modal injection can steer an agent away from intended goals. |
| ASI02 — Tool Misuse | Injected instructions can cause an agent to misuse connected tools. | |
| ASI03 — Identity & Privilege Abuse | Attackers may use injected content to trigger privileged actions through the agent. | |
| Recommendation — Harden goal interpretation so hidden instructions cannot redirect agent objectives. Restrict tool invocation to approved actions after policy checks. Enforce least privilege and separate agent authority from user content. | ||
| NIST AI RMF | GOVERN — GOVERN | AI governance is needed to define accountability for multimodal input risk. |
| Recommendation — Assign ownership for multimodal prompt-injection risk and review controls regularly. | ||
Practitioner Guidance
Why practitioners should care: Multimodal systems expand the attack surface beyond text prompts, so security review has to cover every ingestion path the model can interpret. If the organisation allows the model to extract meaning from images, audio, video or documents, those sources need the same governance discipline as any other untrusted input.
Practitioner note: The key mistake is assuming that “non-text” means “non-instructional.” In practice, the safer assumption is the opposite: any modality that can be interpreted can also be abused.
For the broader control landscape, NIST AI Risk Management Framework is a useful external governance reference for organising AI risk work, and NIST Cybersecurity Framework 2.0 remains a practical umbrella for identifying, protecting, detecting and responding to these failures.