A context window is the maximum amount of text, measured in tokens, that a large language model (LLM) can take into account in a single request, including the answer it writes. Think of it as the model's working memory for one call: the system prompt, the conversation so far, attached documents, tool definitions, tool results and the reply all have to fit inside it. This page explains what counts toward the window, why a bigger window does not automatically mean better answers, how window sizes compare across providers, and how to plan prompts and applications around the limit. It is written for learners first, then for developers and architects who build on LLM APIs. Providers are listed alphabetically; order is never a ranking.
Last checked: 2026-11-15.
This is an independent reference page from NHI Mgmt Group based on public provider documentation; it is not sponsored or endorsed by any provider.
Key takeaways
- The window covers input and output together. System prompt, messages, files, tool definitions, tool results and the generated reply (including any reasoning tokens) all draw on the same budget, unless the provider documents separate limits.
- Window size and maximum output are different numbers. A model can accept around a million tokens yet still cap a single reply at a much smaller figure, such as 128K tokens.
- Claimed length is not effective length. Research such as Lost in the Middle and RULER found that accuracy falls as inputs grow and when the relevant passage sits in the middle of a long prompt.
- Caching changes cost, not capacity. Cached tokens are cheaper on most platforms but still occupy the window.
- Plan the budget explicitly. Count tokens before sending, reserve room for output and reasoning, and trim, retrieve or compact rather than filling the window by default.
What is a context window?
LLMs read and write text as tokens, which are fragments of words, punctuation or other data produced by the model's tokeniser. The context window is the number of tokens the model can attend to at once. Anthropic's documentation describes it as all the text a language model can reference when generating a response, including the response itself, and distinguishes it from the model's training data. Google's Gemini documentation compares it to short-term memory.
Two plain consequences follow. First, anything outside the window does not exist for the model on that request: if an earlier part of a chat is dropped, the model cannot recall it. Second, the model's knowledge of the world comes from training, not from the window; the window is where you put the facts you want it to use right now. See Knowledge Cutoff Explained for the training side of that split.
Token counts depend on the tokeniser, so the same text can produce different counts on different models. Images, PDF pages, audio and video are also converted to tokens on multimodal models, which is why a few large files can consume a large share of the window. Use the provider's token counting tool rather than estimating from word counts.
What counts toward the window?
The exact accounting is provider specific. The table below shows the items that commonly draw on the budget, with what the fetched documentation says.
| Item | Counts toward the window? | What the documentation states |
|---|---|---|
| System prompt and instructions | Yes | Anthropic lists the system prompt, every message, tool results, images, documents and tool definitions. |
| Conversation history | Yes, for every turn you resend | Chat APIs are stateless per request unless a stateful feature is used, so earlier turns are resent and counted again. |
| Tool definitions and tool results | Yes | Anthropic counts both. Large tool outputs in agent loops are a common cause of overflow. |
| Images, PDFs, audio, video | Yes, after tokenisation | Anthropic allows up to 600 images or PDF pages per request on its 1M-token models (100 on 200K-token models). Google notes that video tokenisation affects billing and limits. |
| Generated output | Yes | The reply is part of the window on every provider page checked. |
| Reasoning or thinking tokens | Yes, where the model reasons | OpenAI states reasoning tokens are not visible via the API but still occupy space in the context window and are billed as output tokens. Anthropic states thinking tokens count toward the window and are a subset of max_tokens. |
| Cached prompt tokens | Yes | Anthropic states caching changes what you pay, not whether tokens occupy the window. |
Reasoning models deserve a specific budget. OpenAI recommends reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with its reasoning models, then adjusting once you know how many reasoning tokens your prompts use. Anthropic documents that previous thinking blocks are kept by default on its newer models and are billed again as input in later requests, which grows the history quickly in long conversations. See Reasoning Models Explained.
Context window and maximum output: two different limits
Provider model pages usually show two numbers: the context window and the maximum output per request. The second is often much smaller. A model with a 1M-token window and a 128K-token output cap can read a very long document, but cannot write a reply longer than 128K tokens in one call.
Some providers describe a shared budget and others describe separate budgets. Google's long-context page states that for Gemini 4 (gemini-4-argon) the 1M-token window is shared across input, thinking and output tokens, with output able to reach 1M tokens through long decode, while earlier Gemini 3 and 2.5 models have separate budgets of 1M input and 64K output. Read the model page for the exact rule on the model you use.
How large are context windows today?
The table lists one current row per provider, using the figures each provider publishes on its own documentation page. It is a snapshot, not a full catalogue: see each provider's models page for every model. Where the provider page does not state a figure, the cell says so.
As of 15 November 2026. Providers listed alphabetically.
| Provider | Model(s) | Context window | Maximum output | Source |
|---|---|---|---|---|
| Alibaba (Qwen) | Qwen3.8-27B (open weights) | 262,144 native; up to 1,000,000 with YaRN scaling | Not stated by Alibaba on the model card | Hugging Face model card |
| Anthropic | Claude Fable 5.2, Opus 5.5, Sonnet 5.5, Haiku 5.5 and other current models | 1M tokens | 128K tokens | Context windows docs |
| Cohere | command-a-plus-05-2026 | 128K | 64K | Models docs |
| DeepSeek | deepseek-flash (V4.1-Flash-1017), deepseek-v4-pro | 1M | 384K | Models and pricing |
| Gemini 4 Argon (gemini-4-argon) | 1M tokens, shared across input, thinking and output | 65K, or up to 1M with long decode | Models docs | |
| Mistral AI | Mistral Large 4 (mistral-large-4) | 1M | Not stated by Mistral AI on the model page | Model card |
| Moonshot AI | Kimi K3 series (K3.1 Pro, K3) | 1,048,576 tokens | Not stated by Moonshot AI on the pricing page | Chat pricing docs |
| OpenAI | GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, GPT-6 Luna | 1.05M | 128K tokens | Models docs |
| xAI | Grok 4.7 (grok-4.7) | 500K tokens | Not stated by xAI on the models page | Models docs |
| Z.ai (Zhipu) | GLM-5.3 | 1M tokens | 128K tokens | GLM-5.3 docs |
Older and smaller models on the same platforms often have smaller windows. Anthropic lists 200K tokens for other Claude models, including Claude Sonnet 4.5 (deprecated); Cohere lists 256K for command-a-03-2025 and 8K for command-a-translate-08-2025; Moonshot AI lists 262,144 tokens for its K2 series, which it says will be retired by the end of December 2026.
Why a bigger window does not guarantee better answers
A model that accepts a million tokens does not use every token equally well. Three findings are worth knowing.
- Position matters: Lost in the Middle (Liu et al., TACL 2023) found that models performed best when the relevant information was at the start or end of the input and considerably worse when it sat in the middle, even for models built for long contexts.
- Effective length is shorter than claimed length: RULER (2024) tested 17 long-context models on 13 tasks. Most scored near perfectly on the basic needle-in-a-haystack test but dropped sharply as length grew, and only about half kept satisfactory performance at 32K tokens despite claiming 32K or more.
- Multiple facts are harder than one: Google's long-context page notes strong single-needle retrieval but lower accuracy when searching for several pieces of information at once. Anthropic's documentation names the general effect context rot: accuracy and recall degrade as token count grows.
These results describe the models tested at the time. Newer models may do better, but the practical lesson holds: test with your own documents at the lengths you plan to use, and put the question where the provider recommends. Google's guidance is to place the query at the end of the prompt.
How open-weight models extend context
Many open-weight models are trained at one length and extended at inference time with position-scaling methods. The Qwen3.8-27B card, for example, gives a native length of 262,144 tokens and up to 1,000,000 tokens with YaRN, a RoPE scaling technique, set in the model's configuration. The card warns that common open-source serving frameworks implement static YaRN, which keeps the scaling factor fixed regardless of input length and can affect performance on shorter texts, so it advises enabling the setting only when long contexts are needed. When self-hosting, the window you get also depends on GPU memory, because the key-value cache grows with sequence length. See Running LLMs Locally Explained and Open-Weight vs Closed Models Explained.
What happens when you exceed the window?
Behaviour depends on the provider and API. Anthropic documents that if the input alone exceeds the window, the API returns a 400 invalid_request_error ("prompt is too long"); on Claude 4.5 and newer models, if input plus max_tokens exceeds the window the request is accepted and stops with stop_reason "model_context_window_exceeded" if generation reaches the limit. Other providers return their own errors or truncate according to their documentation. Chat applications built on these APIs usually drop or summarise older turns silently, which is why a long chat can appear to forget early instructions. For the general API error codes, see the LLM API 401, 403 and 429 Errors Guide.
How do context windows affect cost and speed?
- Every resent token is billed: in a multi-turn chat the history is sent again on each turn, so input cost grows with conversation length.
- Long-context pricing tiers: some providers charge more above a threshold. Anthropic states that long-context requests on its 1M-token models are billed at standard pricing, with the exception that prompts over 100,000 tokens on Claude Haiku 5.5 cost more. The OpenAI and xAI model pages checked list flat per-token rates with no long-context tier.
- Caching: reusing a large fixed prefix (a manual, a codebase summary) through prompt or context caching lowers cost on repeat calls. DeepSeek publishes separate cache-hit input prices; Google describes hourly storage charges for cached context.
- Latency: Google notes that longer inputs increase latency, mainly time to first token.
See Model Pricing Explained for the pricing mechanics in full.
Long context, retrieval or compaction: how to choose
There are three main ways to give a model more information than fits comfortably. The choice depends on factual criteria about your data and task, not on which approach is newer.
| Approach | How it works | Fits when | Watch for |
|---|---|---|---|
| Put it all in the window | Send the full documents in the prompt, often with caching | The corpus fits with room to spare, changes rarely and the task needs the whole text (contract review, codebase questions) | Cost per call, latency, lost-in-the-middle effects |
| Retrieval (RAG) | Search an index and insert only the relevant passages | The corpus is much larger than the window or changes often | Retrieval misses, access control on what is retrieved |
| Compaction and context editing | Summarise or clear older turns and tool results as the session grows | Long-running chats and agent loops | Loss of detail in summaries; check what is kept |
Anthropic documents server-side compaction (beta for Claude 4.6 and later models) and context editing features for clearing old tool results and thinking blocks. Retrieval brings its own security questions about who may see what: see the Permission-Aware RAG Guide.
Practical steps for working within the window
- Count before you send: use the provider's token counting endpoint or tokeniser and read the usage field returned with every response.
- Reserve output room: set a maximum output that leaves space for reasoning tokens on reasoning models.
- Front-load stable content: put fixed instructions and reference text first so they can be cached, and the specific question last.
- Trim tool output: return only the fields an agent needs from each tool call.
- Retrieve rather than paste: for large or changing corpora, index and retrieve.
- Compact long sessions: summarise or clear older turns on a schedule rather than waiting for an overflow error.
- Test at real length: evaluate answer quality at the input sizes you will use in production, not only on short samples.
Measures worth tracking: average and 95th percentile input tokens per request, share of requests above 50 percent of the window, overflow error rate, cache hit rate, and answer accuracy on a fixed long-document test set.
Where security fits
Everything in the window can be reflected in the output, so the window is also a data exposure surface: retrieved documents, tool results and secrets pasted into prompts can all leak through responses or logs. The Context Window Data Exposure guide covers classification ceilings, retrieval controls and history filters. Untrusted text placed in the window can also carry instructions: see OWASP LLM01 Prompt Injection.
Practitioner checklist
- Confirm the context window and maximum output tokens for the exact model ID you will call.
- Measure the token count of your typical prompt plus history with the provider's own token counter.
- Check whether long-context pricing or a different rate applies above a token threshold.
- Test retrieval accuracy on your own long documents rather than relying on headline window size.
- Trim or summarise conversation history so each request carries only what the task needs.
- Record the date you checked each limit, as providers change them between versions.
Standards and references
- Anthropic: Context windows (accessed 2026-11-15)
- Cohere: Models overview (accessed 2026-11-15)
- DeepSeek: Models and pricing (accessed 2026-11-15)
- Google: Gemini API long context (accessed 2026-11-15)
- Google: Gemini models (accessed 2026-11-15)
- Mistral AI: Mistral Large 4 model card (accessed 2026-11-15)
- Moonshot AI: Chat model pricing (accessed 2026-11-15)
- OpenAI: Models and Reasoning models guide (accessed 2026-11-15)
- Qwen: Qwen3.8-27B model card (accessed 2026-11-15)
- xAI: Models and pricing (accessed 2026-11-15)
- Z.ai: GLM-5.3 (accessed 2026-11-15)
- Liu et al.: Lost in the Middle: How Language Models Use Long Contexts (accessed 2026-11-15)
- arXiv 2404.06654: RULER: What's the Real Context Size of Your Long-Context Language Models? (accessed 2026-11-15)
Related NHI Mgmt Group resources: How to Read LLM Benchmarks, Model Pricing Explained, Context Window Data Exposure, LLM Security Guide.