When should organisations use multi-modal red-teaming instead of text-only testing?

Why multi-modal red-teaming is the right default for multi-modal models

Text-only testing is only sufficient when the production model can only receive text. Once screenshots, photos, charts, scanned documents, UI captures, or other non-text inputs are part of the real interface, the attack surface changes. The evaluation must cover the same input channels the system will see in production, or it will miss prompt injection, instruction steering, and policy bypass patterns that originate in visual content.

Multi-modal red-teaming is not a separate niche exercise, it is the realistic test for the system as deployed. A model may behave safely on text prompts yet fail when a user embeds instructions in an image, hides malicious intent in a screenshot, or uses layout and visual cues to shift the model’s interpretation. That gap matters most where the model can parse both user text and attached media in one workflow.

For teams building or operating AI systems, the deciding question is simple: does the model have any non-text intake path that can influence its output or actions? If the answer is yes, the red-team scope must include that path. Red Teaming AI Agents for Identity Abuse is useful background where red-team findings may involve delegated authority, but the core principle here is broader, the test must match the interface.

What text-only testing misses in practice

Text-only evaluations miss failures that depend on the model “seeing” something, not just reading something. Examples include hidden instructions inside screenshots, malicious overlays in image-based workflows, text embedded in diagrams, and visual cues that alter classification or content interpretation. In those cases, the model may obey instructions it would have rejected in plain text, or may extract and act on sensitive information that the text harness never exposed.

The practical consequence is blind coverage. If the real product accepts a pasted screenshot of a support ticket, a photographed form, or an uploaded document image, then the model can be influenced through those artifacts even when the visible prompt text looks harmless. That is especially important for assistants that summarize, extract, route, or trigger downstream actions, because a visual input can change not just the answer but the decision path.

Multi-modal testing should also check for mismatches between modalities. A common failure mode is when the text prompt says one thing, but the image contains a conflicting instruction, label, or embedded request that the model privileges instead. Another is where the model extracts sensitive data from a visual object that the text policy never intended to permit. CSA AI Agent Disclosure Accountability Gap whitepaper is relevant here because it highlights how AI systems can fail when the control plane and the user-visible interaction do not match the actual execution path.

How to choose the test scope and stop over-testing the wrong thing

Use multi-modal red-teaming when the production model can ingest any non-text input that may affect behaviour, including screenshots, images, documents, OCR outputs, or combined chat-plus-upload workflows. Text-only testing remains appropriate only when the deployed interface is truly text-only, or when a specific control is being isolated for a narrow regression test. If the user journey includes both text and media, the evaluation should reflect that journey end to end.

The best practice is to test the highest-risk path first: whatever input type can most directly influence tool use, retrieval, summarisation, or action execution. That means prioritising scenarios where visual content can steer the model toward a bad decision, not just scenarios where the model generates an incorrect caption. If the model drives workflows, treat any modality that can shape those workflows as part of the attack surface.

For teams comparing methods, the question is not whether text testing is simpler, it is whether it is representative. MITRE ATLAS adversarial AI threat matrix helps anchor the adversarial mindset, while CSA MAESTRO agentic AI threat modeling framework is helpful when multi-modal inputs can influence coordinated agent behaviour or downstream tool use.

Practitioner Guidance

What to verify: Confirm that your red-team corpus includes every production input channel, not just chat text. If the interface accepts uploads, screenshots, pasted images, or scanned documents, those paths need explicit test cases because they often create the highest-value bypasses.

Decision rule: If a non-text input can change the model’s interpretation, policy outcome, or downstream action, treat text-only testing as incomplete. Reserve text-only testing for narrow cases where the production system genuinely cannot process anything else.

What practitioners underestimate: Visual payloads often matter less because of what they “say” and more because of how they reframe the task, introduce conflicting instructions, or alter extraction behaviour. The failure is usually in interface realism, not in model intelligence.

Practitioner takeaway: Red-team the interface you actually ship, not the simplest prompt path you can test. If multimodal content can steer the system, then multimodal testing is the minimum credible evaluation.

Related resources from NHI Mgmt Group

• When should organisations use manual testing instead of automation for logic flaws?
• Should organisations choose continuous testing or point-in-time red teaming?
• How should organisations compare automated AI red teaming with human-led testing?
• What breaks when organisations rely on one-time AI red teaming instead of continuous retesting?