The context window — the fixed size of one request
In one paragraph
A context window is the total size of a single request, counted in tokens: the system prompt, the conversation, your documents and the reply the model is about to write, all in one fixed space. It is not memory and it is not storage. Nothing carries over between requests except the text the application chooses to resend.
your chat with the AI
One exchange, one window. The default four-line chat is 66 tokens for GPT-4o and about 92 for Claude Haiku 4.5. The window that has to hold it also has to hold every earlier turn, the vendor system prompt and the reply — 128,000 tokens in total for GPT-4o, 200,000 for Claude Haiku 4.5.
What is actually in the window
A chat looks like a conversation and works like a series of independent deliveries. OpenAI puts the mechanism plainly:
each text generation request is independent and stateless
Continuity is the application resending the transcript every turn, and that resent bundle is what has to fit. Anthropic lists the contents:
the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions
The output counts too — the window “holds the conversation history plus the new output Claude generates”.
So the window is not a store of what the model knows, and not a place where things are kept. It is the size of one envelope. Everything in this request goes in it, and nothing survives outside it.
What follows from that
It fills up on its own. Anthropic calls the process progressive token accumulation: each turn, “each user message and assistant response accumulates within the context window, and previous turns are preserved completely”. You do not have to paste anything for the conversation to grow.
Overflow is not graceful. If the input alone exceeds the window, the request fails outright: “If
the input alone already exceeds the model’s context window, the API returns a 400 invalid_request_error (‘prompt is too long’) on every model.” In a chat product the usual
behaviour before that point is to drop the oldest messages — Anthropic notes that chat interfaces
such as claude.ai “can also manage the context window on a rolling ‘first in, first out’ basis”.
Fitting is not using. Everything inside the window is available to the model; that is not the same as being used evenly, and the evidence for that is measured rather than announced. Lost in the Middle moved the answer around inside an otherwise identical 30-document input and watched accuracy fall from 73.4% to 50.5%; Context Length Alone Hurts LLM Performance Despite Perfect Retrieval found performance dropping by 13.9% to 85% on longer inputs that still fitted comfortably. That is a separate idea with its own numbers.
The unit is tokens, not words. The same text costs a different number of tokens in each vendor’s dictionary, and more in German than in English. If you want to know whether something fits, you have to count the way the model counts.
Technical detail: what else quietly occupies the window
Cached prefixes still occupy space — Anthropic states that prompt caching changes what you pay
for those tokens, not whether they count. Extended thinking counts as well: thinking tokens are
a subset of the max_tokens parameter and are billed as output. On some models the
API also injects context-awareness tags such as <budget:token_budget>200000</budget:token_budget>, which the
documentation says you never send yourself.
None of this is visible from a chat interface, which is why the number you can measure — the text you paste — is always a lower bound on what the request actually contains.
Common questions
- 1 Is the context window the same as the maximum length of an answer?
- No. They are separate limits and the output limit is much smaller. Claude Haiku 4.5 reads 200,000 tokens and writes at most 64,000; GPT-4 Turbo reads 128,000 and writes at most 4,096. Confusing the two is the most common wrong explanation for an answer that stops mid-sentence.
- 2 Does a bigger window mean better answers?
- Only up to a point, and what settles that is measurement rather than documentation. Lost in the Middle changed nothing but the position of the answer inside a 30-document input and saw accuracy fall from 73.4% to 50.5%. A later paper, Context Length Alone Hurts LLM Performance Despite Perfect Retrieval, reports performance degrading by 13.9% to 85% as the input grows longer while staying well inside the models' claimed context lengths. Vendors have since started saying the same thing, but a larger window changes what fits, not how well it is used.
- 3 How many pages is a window?
- Anthropic annotates a 200,000-token window as roughly 150,000 words or 680,000 unicode characters, and a 1M-token window as roughly 555,000 words. Pages vary too much to be a unit; that is why nobody states the limit in them.