LLM Side-Channel Attacks: Whisper Leak and Token Leaks

TLS hides what a user and a large language model say to each other, but it does not hide how big each streamed chunk is or when it arrives, and published research shows that is often enough to tell what the conversation is about. Since 2024 a run of papers has turned that observation into practical attacks: token-length inference against ChatGPT and Copilot, timing attacks on speculative decoding, prompt-cache timing across tenants, and Microsoft's Whisper Leak topic classifier tested against 28 models. This guide explains each class of leak, who can exploit it, what providers have changed, and what security teams running or consuming AI services should do. It is written for security architects, AI platform owners and privacy teams who need to decide whether their AI traffic is exposed and which controls actually reduce the risk.

Key takeaways

Encryption is not confidentiality of metadata. Streaming LLM responses send one or a few tokens per network record, so ciphertext sizes and inter-arrival times mirror the plaintext token sequence.
The attacker is a passive network observer. Anyone who can see the encrypted flow (an ISP, a hostile Wi-Fi operator, a nation-state tap, a compromised proxy) can run the classifiers offline without breaking TLS.
Topic inference is the realistic threat today. Whisper Leak reported above 98% AUPRC for topic classification on many of 28 models; full text reconstruction is harder (29% of responses in the 2024 token-length work).
Providers have shipped partial fixes. Random-length padding fields from OpenAI, Microsoft Azure, Mistral and Cloudflare reduce the signal, but the Whisper Leak authors state that none of the mitigations they tested gives complete protection.
Self-hosted inference is your problem to fix. If you run vLLM or a similar server with speculative decoding, shared prompt caches or plain token streaming, the same leaks apply and no provider will patch them for you.

What an LLM traffic side channel is

A side channel leaks information through a by-product of computation rather than through the protected data itself. The general concept, including power, cache and acoustic channels, is covered in the Side-Channel Attacks Guide. LLM traffic side channels are a specific network variant that exists because of how chat and API services deliver output.

Most assistants stream their answer as it is generated, usually as server-sent events over HTTPS, so the user sees text appear token by token. Each event is wrapped in TLS records. Modern TLS cipher suites such as AES-GCM and ChaCha20-Poly1305 are stream-like: the ciphertext length equals the plaintext length plus a fixed overhead. Microsoft's Whisper Leak write-up makes exactly this point, that ciphertext size tracks plaintext size plus a constant. So an observer recording packet sizes recovers a sequence of token lengths, and recording timestamps recovers how long each step of generation took. Neither requires any key material.

Three properties make LLM traffic unusually leaky compared with ordinary web browsing:

Fine granularity: one record per token or small group of tokens gives the observer hundreds of data points per answer, rather than one large page download.
Content-dependent computation: efficiency features such as speculative decoding, parallel decoding and prompt caching make the time per step depend on what is being said.
Predictable language: natural language is highly structured, and an attacker can train a model (often another LLM) to map length and timing sequences back to likely topics or text.

The attack families and what the research showed

The table summarises the four main lines of published work. All are research results disclosed to the affected providers; the figures are the authors' own and depend on their test conditions.

Attack	Signal used	Who can run it	Reported result
Token-length inference ("What Was Your Prompt?", Weiss, Ayzenshteyn, Amit and Mirsky, Ben-Gurion University, USENIX Security 2024)	Size of each encrypted streamed record, giving the length of each token	Passive network observer	Against ChatGPT-4 and Microsoft Copilot (browser and API traffic): 29% of responses accurately reconstructed and the topic inferred for 55%, using an LLM trained to translate length sequences into text
Whisper Leak (McDonald and Bar Or, Microsoft, November 2025)	Packet sizes and inter-arrival times of streaming responses	Passive network observer	28 popular LLMs tested; topic classification often above 98% AUPRC; with 10,000 unrelated conversations per target, 100% precision for many models while catching 5 to 20% of target conversations (the blog post gives 5 to 50% as a projection)
Remote timing attacks on efficient inference (Carlini and Nasr, 2024)	Data-dependent timing from speculative sampling and parallel decoding	Passive observer; active attacker for the boosting variant	Above 90% precision for topic on open-source systems; on ChatGPT and Claude, distinguishing specific messages and inferring the user's language; an active boosting attack recovered PII such as phone and card numbers from open-source systems
Speculative decoding leaks ("When Speculation Spills Secrets", Wei, Abdulrazzag, Zhang, Muursepp and Saileshwar)	Per-iteration token counts or packet sizes, revealing which speculations were accepted	Passive observer	Fingerprinted queries from a set of 50 prompts: REST 100%, BiLD 95.2%, LADE 91.6%, EAGLE 77.6% at temperature 0.3; confidential datastore contents leaked at more than 25 tokens per second; tested including the production vLLM framework
Prompt cache timing (Gu, Li, Kuditipudi, Liang and Hashimoto, Stanford, 2025)	Faster time to first token when a prompt prefix is already cached	Another API customer of the same provider	Cache sharing across users detected in seven API providers including OpenAI, which lets a tenant test whether someone else recently sent a given prompt; also revealed that an OpenAI embedding model is a decoder-only Transformer

Why topic classification matters more than reconstruction

Whisper Leak framed the attack as surveillance: a binary classifier asks whether a conversation is about a sensitive target topic (the test used the legality of money laundering, with 100 phrasings against 11,716 unrelated Quora questions). The authors used LightGBM, a Bi-LSTM and a DistilBERT model extended with size and time tokens, and found accuracy improved with more training data. For an oppressive regime, an employer monitoring a network, or a criminal on a hotel Wi-Fi, knowing that a user asked about a specific medical condition, a whistleblowing procedure or an unannounced merger is already a serious breach of confidentiality, even with no words recovered.

Threat model: who is exposed

Scenario	Exposure	Main lever
Employees using a public chat assistant from home, travel or public Wi-Fi	Highest: traffic crosses networks you do not control and the content is often sensitive	Use providers that have shipped padding; route through corporate network or a trusted VPN; policy on sensitive topics
Applications calling a commercial LLM API from your cloud	Lower: the path is mostly cloud backbone, but egress, proxies and inspection devices see the flow	Enable provider obfuscation options, prefer non-streaming for sensitive back-end calls, keep traffic on private connectivity where offered
Self-hosted inference (vLLM, TGI or similar) serving internal users	Depends entirely on your configuration: no upstream patch	Pad or batch streamed output, review speculative decoding, isolate prompt caches per tenant
Shared multi-tenant inference platform (internal or SaaS)	Cache timing between tenants	Per-tenant cache isolation, or disable cross-tenant prefix reuse

The attacker in the first three rows maps to MITRE ATT&CK T1040 Network Sniffing and, where they control a proxy or access point, T1557 Adversary-in-the-Middle. Note that no TLS interception is needed: a sniffer that only records sizes and times is enough.

What providers have changed

Cloudflare (March 2024): after the Ben-Gurion researchers reported the token-length issue through its bug bounty, Cloudflare added a property named p holding a random-length string to every streamed JSON object from Workers AI, and padded all streaming responses proxied through AI Gateway with variable-length noise. Both changes were automatic, so customers took no action, and the gateway padding protects traffic even when the upstream model provider has not patched.
OpenAI and Microsoft Azure (2025): according to Microsoft, OpenAI added an obfuscation field containing random text of variable length to streaming responses, Azure mirrored it, and Microsoft verified that the Azure mitigation reduces attack effectiveness to a level it no longer considers a practical risk.
Mistral: added a parameter named p with a similar effect.
xAI: listed by Microsoft among providers that had deployed protections at the time of the Whisper Leak post.

Two cautions. First, the Whisper Leak paper itself tested random padding, token batching and packet injection and concluded that each lowers attack success but none provides complete protection. Second, padding is often an API option that clients can switch off to save bandwidth, so check what your SDK and gateway actually send rather than assuming the provider default is in effect.

Mitigations compared

Mitigation	What it hides	Cost	Where to apply
Random padding per streamed chunk	Exact token lengths; reduces size signal	Extra bandwidth; timing still visible	Provider API, AI gateway, self-hosted server
Token batching (send several tokens per record)	Per-token length and some per-token timing	Slightly less smooth streaming	Self-hosted server, gateway
Packet injection (dummy records)	Number and rhythm of records	Bandwidth, implementation effort	Server or gateway
Non-streaming responses	Per-token size and timing; total size and latency remain	User waits for the full answer	Back-end and batch calls, high-sensitivity flows
Iteration-wise token aggregation (for speculative decoding)	Which speculations were accepted	Some latency benefit lost	Self-hosted inference engines
Disable speculative or parallel decoding for sensitive tenants	Content-dependent timing	Throughput and latency	Self-hosted inference engines
Per-tenant prompt cache isolation	Cross-tenant cache hits	Lower cache hit rate	Multi-tenant inference platforms
Trusted network path (corporate network, private link, trusted VPN)	Removes the hostile observer from the local segment; the flow is still visible further along	Network design and cost	Users on untrusted networks, cloud egress

Decision guidance

Consumers of public assistants (most organisations): you cannot change the provider's streaming design, so your levers are provider choice, network path and usage policy. Confirm in vendor due diligence whether streaming padding is on by default and whether it applies to the consumer app, the enterprise workspace and the API. See the ChatGPT Enterprise Security Guide and the Azure OpenAI and Azure AI Foundry Security Guide for the wider controls on those services.
API builders: for server-to-server calls that do not need a live typing effect, non-streaming is the simplest control. Where streaming is needed, put an AI gateway in front that pads output, and keep obfuscation options enabled in client code. See the AI Gateway and LLM Gateway Guide.
Self-hosters: treat inference servers like any other crypto-sensitive service. Padding and batching in the serving layer, a deliberate decision on speculative decoding, and cache isolation per tenant or per trust level. See the Self-Hosted LLM Security Guide.
Small organisations: focus on policy and provider choice; you are unlikely to be a target of bespoke classifiers, but staff on public Wi-Fi discussing regulated data is a cheap win to address.
High-risk users (journalists, legal, M&A, health, people in hostile jurisdictions): assume topic leakage is possible on any network you do not trust, and route sensitive AI use through managed, padded and preferably non-streaming paths.

How to implement

Inventory AI traffic paths. List every assistant, API integration, agent and self-hosted model, whether it streams, which network it crosses, and who could observe it. Your AI gateway or proxy logs and the AI Security Posture Management (AI-SPM) Guide help build the list.
Classify by sensitivity. Mark flows that may carry regulated data, privileged legal or deal information, health or security incident details. These get the strongest controls first.
Check provider mitigations. For each external model, record whether streaming padding is on, whether it can be disabled by clients, and the date you verified it. Ask in vendor questionnaires; vendors that patched Whisper Leak will know the answer.
Fix the client side. Make sure SDK wrappers and gateways do not strip padding fields, and switch sensitive back-end calls to non-streaming.
Harden self-hosted servers. Add random padding or token batching to streamed responses, evaluate the latency cost of iteration-wise aggregation or disabling speculative decoding for sensitive workloads, and scope prefix caches per tenant.
Test before and after. In a lab, capture traffic for a fixed prompt set with and without the mitigation, plot record sizes and gaps, and confirm the token-length pattern is no longer visible. The public Whisper Leak code and data can support an authorised internal test; keep it to your own systems.
Roll out in rings. Enable padding or batching for an internal pilot group, watch latency and bandwidth metrics and user feedback, then widen. Roll back by configuration flag if latency breaches your SLO, and record the residual risk if you do.
Update policy and training. Add a line to the acceptable use policy on sensitive topics over untrusted networks and approved assistants only, aligned with your AI Acceptable Use Policy Template.

Detection and measurement

A passive side-channel attack leaves no trace on your systems, so detection focuses on exposure, not on catching the attacker.

Exposure monitoring: log at the AI gateway whether each streamed response carried padding, and alert when a client or route bypasses the gateway to reach a model API directly.
Configuration drift: track speculative decoding, prefix caching and streaming settings on self-hosted servers as configuration items, with change alerts.
Cache timing self-test: periodically run the prompt-caching audit approach from the Stanford paper against your own multi-tenant platform to confirm no cross-tenant hits.

Metrics that show the controls are working:

Percentage of sensitive AI flows using padded or non-streaming responses (target 100%).
Percentage of external model providers with verified side-channel mitigation and a verification date under 12 months old.
Number of AI API calls that bypass the approved gateway per week (target trending to zero).
Self-hosted inference endpoints with cache isolation per tenant (target 100% of shared clusters).
Classifier accuracy in your own lab test after mitigation, compared with the unmitigated baseline.

Common failure patterns

Assuming TLS 1.3 solves it. TLS 1.3 supports record padding, but common implementations do not pad application data by default, and the papers above attacked traffic over standard HTTPS. See the TLS Server Configuration Baseline Guide for what TLS does protect.
Stripping padding for efficiency. Custom gateways and SDK forks that normalise responses or drop unknown fields can remove obfuscation fields silently.
Self-hosting with defaults tuned for speed. Teams enable speculative decoding and shared prefix caching for throughput without a security review of what timing they expose.
One shared cache for all tenants. Multi-team inference platforms that reuse prompt prefixes across trust boundaries recreate the cross-user cache timing leak internally.
Ignoring the network path in vendor reviews. AI vendor assessments ask about retention and training on data, but rarely about metadata leakage in transit.

Framework mapping

Framework	Reference	How it applies
NIST SP 800-53 Rev. 5	SC-8 Transmission Confidentiality and Integrity	Confidentiality of AI traffic includes metadata that reveals content; padding and batching support this control
NIST SP 800-53 Rev. 5	SC-31 Covert Channel Analysis	Timing and storage channel analysis is the established discipline for this class of leak; apply it to inference servers
NIST SP 800-53 Rev. 5	SA-8 Security and Privacy Engineering Principles; RA-3 Risk Assessment	Review efficiency features (speculative decoding, caching) for information leakage before enabling
NIST SP 800-53 Rev. 5	AC-4 Information Flow Enforcement; SC-7 Boundary Protection	Route model traffic through approved gateways and private paths
NIST CSF 2.0	PR.DS-02; ID.RA-01	Protect data in transit; record side-channel exposure as an identified vulnerability
MITRE ATT&CK	T1040 Network Sniffing; T1557 Adversary-in-the-Middle	The observer positions an attacker needs
MITRE ATLAS	AML.T0040 AI Model Inference API Access	The cache timing attack relies on ordinary inference API access as a co-tenant
OWASP Top 10 for LLM Applications 2025	LLM02 Sensitive Information Disclosure	Topic and content leakage is a disclosure outcome; see the OWASP LLM Top 10 LLM02 Sensitive Information Disclosure Guide

Practitioner checklist

List every streamed AI flow, the network it crosses and its data sensitivity.
Record, with a verification date, whether each model provider pads streamed responses.
Confirm SDKs and gateways preserve obfuscation or padding fields end to end.
Switch sensitive back-end model calls to non-streaming where latency allows.
Put an AI gateway that pads streamed output in front of providers that do not.
On self-hosted servers, add padding or token batching and review speculative decoding.
Isolate prompt and prefix caches per tenant or trust level on shared inference platforms.
Run a lab capture before and after mitigation to confirm the length pattern is gone.
Add AI use on untrusted networks to the acceptable use policy and staff awareness.
Include metadata leakage in transit in AI vendor due diligence questions.

LLM Side-Channel Attacks Guide: Whisper Leak, Token-Length and Timing Leaks in Encrypted AI Traffic