ASCII Smuggling and Invisible Prompts: Detecting and Stripping Hidden Unicode in LLM Inputs

ASCII smuggling hides instructions or data inside text using Unicode characters that render as nothing on screen but are still read by a large language model. The usual carrier is the Unicode Tags block, U+E0000 to U+E007F, whose characters mirror printable ASCII one for one, so any English sentence can be written in a form a human reviewer cannot see. The same trick works in reverse: a model can be told to encode stolen data invisibly inside a link or a reply. This guide explains what the technique is, how the encoding works, why it matters for assistants and agents, and how to detect and strip it in an input pipeline. It is written for engineers building LLM applications and for security teams testing them. It goes deeper on the Unicode mechanics than the Prompt Injection Guide, which covers the wider attack class.

Key takeaways

• It is a hiding technique, not a new attack class. ASCII smuggling is a way to deliver prompt injection (MITRE ATLAS AML.T0051) and to exfiltrate data, filed by ATLAS under AML.T0068 LLM Prompt Obfuscation.
• The Tags block is the main carrier. Each tag character is its ASCII counterpart plus 0xE0000, so "A" (U+0041) becomes U+E0041, and the result is invisible in most interfaces.
• Strip on the way in and the way out. Remove tag characters before text reaches the model and again before model output is rendered, logged or sent onward.
• One pass is not always enough. AWS showed that in UTF-16 runtimes such as Java, removing characters can let surrogate halves recombine into new tag characters, so repeat until the text stops changing.
• Stripping reduces risk, it does not end prompt injection. Visible injection, other encodings and hidden HTML still work, so keep least privilege and output controls in place.

What it is

The term ASCII smuggling was popularised by security researcher Johann Rehberger in January 2024, when he published a proof of concept against ChatGPT and a small encoder and decoder tool called ASCII Smuggler. His point was simple: the Unicode Tags block mirrors ASCII, models trained on web text have learnt to read it, and user interfaces show nothing. A paragraph can therefore carry a second, hidden paragraph that only the model sees.

The Unicode Standard (code charts, version 18.0) defines the Tags block as U+E0000 to U+E007F. U+E0001 LANGUAGE TAG is deprecated and the chart states that using tag characters to convey language tags is strongly discouraged. U+E0020 TAG SPACE to U+E007E TAG TILDE mirror the printable ASCII range, and U+E007F CANCEL TAG is a terminator. The block survives because emoji use it: the flag of England is a black flag emoji followed by tag letters and a cancel tag, which is why some flags break when tags are stripped.

"Invisible prompt" is the broader term. It covers any instruction a human cannot see but a model ingests, including zero-width characters, white or zero-size text, hidden HTML and text in image metadata. ATLAS AML.T0068 lists those rendering tricks alongside encodings. This guide concentrates on the Unicode carriers, because they pass through copy and paste, email, chat, documents, code and APIs intact.

How it works

The encoding is arithmetic. Take each printable ASCII character, add 0xE0000 to its code point, and append the result to an innocent visible string. In UTF-8 each tag character takes four bytes; in UTF-16 (Java, JavaScript, .NET strings) each becomes a surrogate pair. Most fonts have no glyph and most renderers treat the characters as ignorable, so the text looks normal in a browser, a mail client or a chat window. A tokenizer still turns them into tokens, and capable models decode them.

Two attack directions follow from this.

• Inbound (hidden instructions): an attacker plants tag text in content the model will read: an email, a shared document, a web page, a ticket, a code comment or a tool description. When a user asks the assistant to summarise it, the model follows the hidden instruction. The AWS Security Blog example is an email assistant told, invisibly, to delete the inbox. This is indirect prompt injection (AML.T0051.001) with a better disguise.
• Outbound (hidden data): an injected instruction tells the model to encode sensitive data as tag characters, often inside a hyperlink. Rehberger's August 2024 disclosure against Microsoft 365 Copilot chained automatic tool invocation (searching the user's mail) with a rendered link whose invisible part carried sales figures and an MFA code to an attacker's server when clicked. ATLAS records the pieces as AML.T0086 Exfiltration via AI Agent Tool Invocation and AML.T0077 LLM Response Rendering. He reported the issue in January 2024; by July the reproduction no longer worked, and Microsoft did not publish details of the fix.

Human review does not help much. The same research notes that a person can approve, copy or forward text that looks clean, so a human in the loop can pass the hidden payload along.

Why it matters

Most LLM defences assume someone can see the malicious text: a reviewer reading a document, a moderator scanning a prompt log, a developer reading a pull request. Invisible carriers break that assumption in four places.

• Agents with tools: an assistant that can send mail, call APIs or browse turns a hidden instruction into an action under the user's identity or a service account. Excessive agency (OWASP LLM06) is what makes the hiding worth the effort.
• Logs and audit: many log viewers render tag characters as nothing, so an investigator reading the stored prompt sees a benign request. Store a raw or escaped copy.
• Code and supply chain: the GlassWorm campaign hid malicious JavaScript in VS Code extensions using invisible Unicode, so the code looked like blank lines to reviewers and to static analysis. AI coding assistants read the same files and comments. See the GlassWorm VS Code Extension Worm Guide.
• Keyword guardrails: filters that match words like "ignore previous instructions" miss text written in tags or split with zero-width joiners. Microsoft's Prompt Shields documentation lists encoding attacks as a category, but its page does not describe invisible characters specifically, so test your own guardrail rather than assume coverage.

Detecting and stripping hidden Unicode

Put a normalisation step at every boundary where untrusted text enters or leaves the model. OWASP ASVS 5.0 requirement V1.1.1 asks that input is decoded into a canonical form once, before validation; apply the same principle to Unicode.

1. Inventory the entry points. User prompts, retrieved documents (RAG), email and calendar connectors, web fetch tools, MCP tool descriptions and outputs, file uploads, and system prompts loaded from repositories.
2. Log before you block. For two weeks, count tag, zero-width, bidi and variation selector code points per source without changing the text. This shows legitimate use (flags, Indic scripts that need U+200C and U+200D, Arabic and Hebrew bidi) and avoids breaking real users.
3. Strip tag characters by default. Remove U+E0000 to U+E007F. In Python, which works on code points, a single pass that also drops lone surrogates (U+D800 to U+DFFF) is sufficient according to AWS. In Java and other UTF-16 runtimes, repeat the removal until the string no longer changes and remove orphaned surrogates.
4. Apply a context policy to the rest. Remove zero-width and bidi controls from prompts and tool arguments; keep U+200C and U+200D only where the language or emoji sequence needs them. Decide whether flag emoji matter enough to allowlist the exact tag sequences.
5. Record what was removed. Log source, count and a decoded preview (tags minus 0xE0000) to the security log, not to the model. A document with 400 hidden ASCII characters is an incident signal, not noise.
6. Sanitise output too. Strip the same code points from model responses before rendering, especially inside URLs and Markdown links, and block auto-rendered images and links to unapproved domains (OWASP LLM05). See the OWASP LLM05 Improper Output Handling Guide.
7. Test and keep a rollback. Add tag-encoded and zero-width test strings to the red team suite and CI, run in log-only mode on a pilot, then enforce. Keep the filter behind a feature flag so a false positive on a language can be turned off per source in minutes.

Block or sanitise? Sanitising (removing characters, then processing) keeps the user experience and is the right default for documents and mail. Blocking (rejecting the request) suits high-risk agent actions and API inputs where no legitimate caller sends tag characters. AWS notes that Amazon Bedrock Guardrails denied topics can block, but not sanitise, tag characters, which is why it pairs them with a separate sanitiser. Small teams can start with a single shared sanitiser library; larger organisations should enforce it centrally in an AI gateway so every application inherits it. See the AI Guardrails and AI Firewalls Guide.

Measure it: track (1) the share of LLM entry points behind the sanitiser, (2) hidden characters removed per 10,000 requests by source, (3) documents or emails with more than a threshold of tag characters, (4) red team pass rate for invisible payloads, inbound and outbound, and (5) false positive reports per language after enforcement.

Common misconceptions

• "If I cannot see it, the model ignores it." Models read tokens, not pixels. Tag characters tokenise, and current models decode them.
• "Unicode normalisation (NFC or NFKC) removes them." Normalisation forms do not delete tag or zero-width characters; you need an explicit removal step.
• "A guardrail product handles it." Some do, some block rather than clean, and some match only visible text. Prove it with tag-encoded test cases before relying on it.
• "Human approval stops it." The approver sees clean text and can forward the hidden payload. Display a warning or escaped view when hidden characters are present.
• "The vendor fixed it." Fixes such as the Copilot change addressed specific rendering paths. The underlying prompt injection risk remains, as the researcher noted, so your own application needs its own control.

Common failures in practice: sanitising user prompts but not retrieved documents or tool output; stripping once in a UTF-16 runtime; logging the rendered prompt so investigators see nothing; stripping zero-width joiners globally and breaking Indic and emoji text, which then gets the filter switched off; and allowing the model to emit links to any domain.

Related concepts

Neighbouring topics: homoglyph and mixed-script attacks (look-alike characters), Trojan Source bidi reordering in code (CVE-2021-42574, with the homoglyph variant CVE-2021-42694), hidden HTML and CSS text in web pages, and image-borne instructions for multimodal models. Testing approaches are in the AI Red Teaming Guide.

Where identity fits: the damage from a hidden instruction is bounded by what the agent's identity can do. Give assistants scoped, short-lived credentials, separate read and write tools, and require confirmation for actions that send data out. A tag-encoded instruction that reaches an agent with read-only access to one mailbox is an annoyance; the same instruction reaching an agent with a broad API key is a breach.

Practitioner checklist

• List every path by which untrusted text reaches a model, including tool descriptions and tool output.
• Run a log-only census of tag, zero-width, bidi and variation selector code points per source.
• Strip U+E0000 to U+E007F on input and output; loop until stable in UTF-16 runtimes and remove lone surrogates.
• Apply a per-context policy for zero-width joiners and bidi controls rather than a blanket ban.
• Store raw or escaped prompts in security logs and alert on documents with large hidden payloads.
• Restrict rendered links and images in model output to approved domains.
• Add inbound and outbound invisible payload cases to red team and CI tests.
• Scope agent credentials so a hidden instruction cannot reach sensitive actions without confirmation.

Standards and references

• Unicode Consortium: Tags code chart (Unicode 18.0)
• MITRE ATLAS: LLM Prompt Obfuscation
• OWASP: Top 10 for LLM Applications 2025
• OWASP: Application Security Verification Standard 5.0
• AWS Security Blog: Defending LLM applications against Unicode character smuggling
• Embrace The Red: Hiding and finding text with Unicode tags
• Embrace The Red: Microsoft 365 Copilot prompt injection, tool invocation and data exfiltration using ASCII smuggling
• Trojan Source: Invisible vulnerabilities

Related NHI Mgmt Group resources: Prompt Injection Guide · Unicode Security · LLM Security Guide · AI Guardrails and AI Firewalls Guide · MCP Security Guide