What is a context window?

It is not memory

A model does not remember your last conversation. What creates the illusion of memory is that the application resends the earlier messages with each new request. The window is working space for one call, not storage.

This has a direct consequence people meet as a surprise: a long conversation gets more expensive with every message, because the whole history is re-sent and re-charged each time.

A full window is not a well-used one

A large window is an allowance, not a target. Models attend unevenly across a long context — material in the middle of a very long input is more likely to be treated lightly than material at either end.

In practice a focused ten pages usually beats an unfocused two hundred. Retrieval exists precisely to choose those ten pages rather than sending everything and hoping.

How it drives cost

Almost all providers bill by token, input and output separately, and input is usually cheaper. A large document attached to every message in a long conversation is charged on every message, not once.

This is the single most common cause of a surprising bill, and the fix is structural: attach what the question needs rather than everything that might be relevant.

Questions people ask

Is a bigger context window always better?

It removes a limit, which is good, but it does not improve how well the model uses what it is given. A bigger window mostly buys you the ability to make a more expensive mistake.

What is a token?

Roughly a word fragment. English text runs around three-quarters of a word per token on average, so a thousand tokens is about seven hundred and fifty words — but this varies by language, and non-Latin scripts often use more tokens per word.

What happens when I exceed it?

The request fails, or the application silently drops the oldest messages. The second is more common and more confusing, because the model appears to forget something you said.