Visual prompt injections expose a new control gap in multimodal AI

At a glance

What this is: This is Lakera’s analysis of visual prompt injection, showing that malicious instructions embedded in images can redirect multimodal model behaviour after the image is processed.

Why it matters: It matters because multimodal AI now expands the attack surface beyond text, forcing practitioners to govern model action scope, not just input filtering, across agentic AI and broader AI security programmes.

Context

Visual prompt injection is a multimodal attack pattern in which an image carries hidden instructions that a model may follow when it reads or describes the image. The security issue is not the image alone, but the way a model can be induced to treat untrusted visual content as instructions rather than data.

For identity and access teams, the issue sits inside AI governance as a control-boundary problem: once a model can consume images, it can be manipulated into changing outputs, suppressing content, or acting outside the operator’s intent. That makes the relevant question what the system can do after interpretation, not whether the prompt looked safe at submission time.

Lakera’s examples are deliberately provocative, but the underlying condition is typical of early multimodal deployments: the model is trusted to reason over user-supplied content without a sufficiently strict boundary between perception and instruction.

Key questions

Q: What breaks when a multimodal model treats image text as instructions?

A: The model can override the user’s intended task, suppress visible content, or rename subjects because it has blurred the line between data and directive. That failure matters most when downstream workflows trust the model’s output as if it were policy-bound rather than influenceable by the image itself.

Q: Why do multimodal models need tighter governance than text-only chatbots?

A: Because visual inputs expand the path by which an attacker can influence model behaviour. The risk is not only bad answers, but altered perception that can change what the model is willing to say or do after it interprets the image.

Q: How should security teams test for visual prompt injection in multimodal AI systems?

A: Test the full input path, not just the model output. Use adversarial images with hidden text, overlays, and instruction-like captions, then verify whether the model changes its answer, ignores visible subjects, or triggers downstream actions. The goal is to prove that malicious image content cannot steer protected behaviour.

Q: When does visual prompt injection become an agentic AI governance issue?

A: When the model can do more than describe the image and can instead retrieve data, call tools, or trigger workflows. At that point, a visual injection can move from misleading output into operational impact, so action controls matter as much as input controls.

Technical breakdown

How image-based instructions bypass text-only prompt controls

Visual prompt injection works when a multimodal model extracts text or meaning from an image and then treats that content as part of the instruction stream. The attacker does not need to break the model’s weights or infrastructure. Instead, they exploit the model’s willingness to merge untrusted visual content with task instructions. That is why a captioning or OCR-like capability can become a control weakness when the downstream policy assumes the model will only answer about the image, not obey embedded directives. Practical implication: treat image interpretation as instruction intake, not as passive content rendering.

Practical implication: Separate visual parsing from instruction authority before allowing image-derived text to influence outputs or actions.

Why multimodal systems create a broader execution boundary

A text-only prompt injection usually aims to change what the model says. A multimodal injection can change the model’s perception of the scene first, which then changes what it is willing to say or do. That distinction matters because the control boundary has moved from input validation to execution governance. Once images are admissible inputs, the model may be persuaded to ignore a subject, rename a person, or suppress nearby entities in the output. Practical implication: govern the execution layer around the model, because filtering the prompt alone does not constrain the model’s behaviour after visual interpretation.

Practical implication: Add output and action policy checks that constrain model behaviour after image understanding, not just before ingestion.

What visual prompt injection means for agentic AI security

In agentic AI systems, visual prompt injection becomes more than a content problem because the model may also retrieve data, invoke tools, or trigger workflows after interpreting the image. That creates an execution risk: an apparently harmless image can become the trigger for a wider chain of model behaviour. This is where agentic AI governance and NHI thinking intersect, because the model is operating with delegated authority in a runtime context. Practical implication: classify image-capable AI as an execution surface and apply policy boundaries to any tool use that follows vision input.

Practical implication: Require policy gates around tool use, retrieval, and workflow triggers that occur after multimodal interpretation.

Threat narrative

Attacker objective: The attacker wants the model to follow embedded visual instructions instead of the legitimate task, altering outputs or suppressing competing content.

1. Entry occurs when a user supplies an image that contains hidden textual instructions or manipulative visual cues.
2. The model ingests the image and interprets the embedded instruction alongside or instead of the user’s original intent.
3. The model then alters its output or suppresses content according to the malicious instruction, producing unintended behaviour.
4. Impact is reached when the manipulated output affects trust, decision-making, or any downstream workflow that depends on the model’s response.

Breaches seen in the wild

• EchoLeak (Microsoft 365 Copilot) 2025: A crafted email could make Microsoft 365 Copilot leak data from its context with no click, a zero-click prompt injection fixed as CVE-2025-32711.
• ForcedLeak (Salesforce Agentforce) 2025: A critical Agentforce flaw let attackers steer an AI agent through a Web-to-Lead form to leak CRM data via an expired allowlisted domain.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Visual prompt injection is a control-boundary failure, not a content-moderation edge case. Once a model can read images, the security problem becomes whether visual content can influence instruction priority. That moves the issue into AI governance, where operators must define what the model is allowed to do after perception, not only what it may accept as input. The practitioner conclusion is that multimodal safety has to be treated as execution governance.

Prompt hygiene is insufficient once the model can infer instructions from images. The old assumption was that malicious intent entered through text and could be filtered as text. Visual prompt injection breaks that premise because the instruction arrives disguised as data. The implication is that multimodal deployments need explicit separation between perception, instruction parsing, and action authority.

Execution scope is the new attack surface: a model that can see, retrieve, or act can be steered through any channel that reaches its reasoning layer. This is why the risk profile changes when AI moves from chat to image-enabled workflows and then into tool use. Practitioners should treat every post-interpretation action as part of the control perimeter, not as a harmless extension of the prompt.

Agentic AI increases the blast radius of visual prompt injection. In a simple assistant, the harm may stop at a misleading answer. In an agentic workflow, the same manipulated interpretation can cascade into retrieval, tool calls, or workflow changes. That means governance has to account for delegated authority after multimodal input, not just for model refusal behaviour.

Visual prompt injection exposes the need for named policy around untrusted content classes. Images, screenshots, and rendered text should not be treated as equivalent to trusted operator instructions. Organisations that collapse those categories create a path for hidden directives to travel through normal business workflows. The practitioner conclusion is to define explicit trust tiers for multimodal inputs.

What this signals

Visual input has become an instruction path. Multimodal models that consume screenshots, adverts, and other rendered content need explicit trust boundaries because the image itself can carry hidden operator intent. The operational question is no longer whether the model can read the image, but whether it should be allowed to act on anything it finds there.

Agentic workflows magnify the consequence of visual prompt injection. A manipulated description is inconvenient; a manipulated description that can trigger retrieval or tool use is a governance failure. Programmes that already treat untrusted text as hostile should extend the same posture to images, OCR output, and any post-vision action channel.

For practitioners

• Define separate trust tiers for image inputs Classify screenshots, user-generated images, and external visual content as untrusted instruction-bearing inputs unless they are explicitly verified. Make that classification drive whether the model may summarise, infer, or act on the content.
• Constrain post-vision actions Require explicit policy checks before any retrieval, tool call, or workflow trigger that follows image interpretation. The control objective is to stop image-derived instructions from becoming operational actions without review.
• Test multimodal refusal boundaries Red team for cases where the model is asked to ignore a subject, rename a person, or suppress nearby content because those are the behaviours visual prompt injection is designed to manipulate.
• Separate OCR from instruction handling Do not let extracted text from images flow directly into the same pathway that carries user prompts or system directives. Keep parsing and instruction authority distinct so hidden text cannot inherit the wrong privilege.
• Monitor for abnormal output suppression Watch for responses that omit visible entities, refuse to mention nearby content, or mirror embedded visual directives. Those are practical signs that the model is treating image content as instruction rather than evidence.

Key takeaways

• Visual prompt injection shows that multimodal AI can be manipulated through image content, not just text, which expands the attack surface for model behaviour.
• The practical risk is control loss after perception, because an image can change what the model says, suppresses, or does next.
• Security teams need to govern untrusted visual inputs, separate parsing from authority, and test whether post-vision actions stay inside policy.

Key terms

• Visual prompt injection: A visual prompt injection is an attack where malicious instructions are embedded in an image so a multimodal model follows them during interpretation. The risk is not limited to bad text output. It can also influence recognition, suppression, or downstream action when the system treats visual content as instruction-bearing input.
• Multimodal model: A multimodal model processes more than one input type, such as text and images, in the same system. That broader input surface increases the trust problem because the model may not naturally distinguish user data from attacker-controlled instructions unless the application adds explicit governance around those inputs.
• Execution boundary: The point at which an authorised task turns into a real system change, such as writing data, deleting records, spending money, or invoking a downstream tool. In AI governance, controlling the execution boundary matters more than simply approving access, because harm occurs when actions are allowed to complete unchecked.
• Instruction Hierarchy: Instruction hierarchy is the order of authority a model applies when interpreting context, system prompts, role messages, and user input. When attackers can influence that hierarchy through templates or wrappers, they can steer behaviour without needing to change the model itself.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.