Why my document is too long for the chat

What’s actually happening

The limit you hit is not a file size and not a page count. It is a token count, and tokens are pieces of text of uneven length — a common word is one, a rare name is three or four, and the same sentence costs a different number under each vendor’s dictionary.

You can see the spread in the demo above. Four short lines are 66 tokens for GPT-4o, 67 for GPT-4 Turbo, and about 122 for Claude Opus 5, because Anthropic’s current tokenizer splits text more finely and Anthropic has not published it, so our count for it is an estimate and says so. Nothing about the text changed. The measuring stick did.

Whatever the count comes to, it has to fit into one fixed space along with everything else in the request. OpenAI states the rule directly:

Your document is not measured on its own; it is measured together with the conversation around it and the reply that has not been written yet.

When it does not fit, the behaviour is worth knowing precisely, because two different things get confused here.

If the input alone is larger than the window, the request fails before generation starts. Anthropic documents it in one line:

That is the blunt refusal — the message that appears the moment you hit send.

If the input fits but there is no room left for a long enough answer, that is a different failure with a different feel, and it is covered in why the answer stops in the middle.

For scale, the vendors publish their own equivalents. Anthropic annotates a 200,000-token window as roughly 150,000 words or 680,000 unicode characters, and a 1M-token window as roughly 555,000 words or 2.5M characters. Those are the vendor’s figures for the vendor’s tokenizer, and they are a much better sanity check than counting pages.

What to do about it

Measure before you paste. Context fit takes text, Markdown or a PDF and gives you the token count against each model’s window, in the browser. Ten seconds of counting replaces a guess about whether an eighty-page contract will go through.

Send the part you are actually asking about. Three relevant pages beat eighty pages of which three are relevant — not only for the limit, but because position inside a long input changes how well material is used.

Split by structure, not by size. One request per chapter, section or party keeps each answer attached to the material it came from. Splitting mid-sentence into equal chunks produces answers about fragments.

Check what a scan really contains. A photographed or scanned PDF has no text layer at all. If your tool reports next to nothing, the document was never read — that needs OCR before any of this applies.

Watch the language. The same text in German or Finnish costs noticeably more tokens than in English, because the dictionaries were fitted to what appears most often. If a document is close to the line, the translation may be over it. Token counter shows the difference on your own text.