What is a context window?

A language model doesn't read an entire document without limit. Before responding, it converts all the relevant text (the system prompt, the conversation history, any attached document) into units called tokens, and it can only take a maximum number of those units into account in a single request. That maximum is the context window.

A token doesn't equal exactly one word: it can be a whole word, part of a word, or a punctuation mark, depending on how the model splits the text. That's why the context window is always measured in tokens, not words or characters, and the conversion between the two varies noticeably by language and by model.

The context window shouldn't be confused with the model's training memory. Training is the enormous corpus of text used to adjust the model's parameters, already fixed before the conversation exists. The context window is different: it's the working memory of that specific conversation, the space that holds what has been said so far and what the model is about to generate next. A model can know something from its training and still be unable to use it in a response if that information never actually entered that specific conversation's context window, and only ever sat somewhere in the training data.

In a multi-turn conversation, each user message and each model response accumulate within the context window, and earlier turns are kept in full unless the application deliberately trims them. Each turn has an input phase, which includes all previous history plus the new message, and an output phase, the response the model generates, which becomes fully part of the history for the next turn and counts toward it as well.

Everything counts toward the limit: the system prompt, every message in the history (including tool results, images, or attached documents), the definitions of the available tools themselves, and the response the model generates, including any internal reasoning it produces before the visible response.

When a conversation approaches the window's limit, there are two main ways out. One is that the application trims or summarizes the oldest parts of the conversation to make room for new ones, a technique usually called compaction. The other is that, if the request already exceeds the limit before generating anything, the system rejects it outright instead of trying to process it halfway.

An important point that changes over time: the exact size of the context window isn't a fixed property of language models in general. It's a figure each provider sets for each specific model, and it has grown with every new generation. That's why any specific number read today about a model can become outdated in a short time, and it's always worth checking it in the relevant provider's current documentation at the actual moment of use, not in an old article.

The context window explains why a very long page doesn't fit whole into an AI answer. When an AI-powered search system pulls content from several pages to build an answer, every passage it includes takes up part of that limited window, and the system has to choose which passages fit and which get left out. A page structured into clear sections, with the most citable information concentrated into self-contained passages, has a better chance that one of those passages fits whole and gets used as is.

This connects directly to chunking, the technique of splitting a long document into manageable pieces before indexing it: chunking decides how a document gets cut, and the context window decides how many of those cuts fit into a given answer. They're two halves of the same problem, not two entirely separate topics, even though they often get discussed apart.

Also, a bigger context window isn't automatically better. Anthropic's own documentation warns of a phenomenon called context rot: the more text a conversation accumulates, the harder it gets for the model to maintain accuracy and recall of the relevant details. That's why choosing well what actually goes into the context matters at least as much as how much space is available overall.

• Structure long content into sections with clear headings, so a specific passage can still be understood even if it's extracted without the rest of the page.
• Never assume a large context window automatically fixes poorly organized content: context rot demonstrably affects even the widest available windows.
• Put the most citable information (definitions, specific figures, steps in a process) in self-contained paragraphs, not spread across several sentences that depend on each other.
• If you're working with a language model's API, check how many tokens your request consumes before sending it, instead of finding out through a limit-exceeded error.
• Verify a model's context window size in the provider's current documentation, not in a year-old article.
• In long conversations with an AI assistant, summarize or close out topics that are already resolved instead of letting the full history keep growing unchecked.

• Citing a model's token count as if it were a universal figure, when each provider sets it per model and changes it with every version.
• Confusing the context window with the model's training capacity, as if it remembered everything it was trained on within every single conversation.
• Assuming that packing as much content as possible into the context improves the response, ignoring that an overloaded context degrades accuracy.
• Writing web content as one continuous block with no structure, which makes it harder for an isolated passage to make sense if it's the only thing that fits in the window.