What is LLM "context"?

When people talk about the "context window" in large language models like ChatGPT or Claude, they are referring to the specific amount of information you feed into the model to narrow down its statistical guesses. Think of the LLM as a massive black box full of numbers derived from reading books, Reddit comments, and blog posts. It doesn't understand meaning in the human sense; instead, it calculates which tokens—chunks of text roughly three-quarters the length of a standard word—tend to appear together.

Context is the tool you use to reduce that search space. If you ask the model to rewrite a report about the phrase "all right, all right, all right," the context determines the output. If the surrounding data suggests a musical reference, the model will likely lean toward Andre 3000 and the song "Hey Ya!" because that phrase frequently appears near his name in training data. If the context points toward film, it might connect to Matthew McConaughey in Dazed and Confused. The model is simply following probability patterns it has learned from millions of examples.

This mechanism also applies to instructions like "respond like Shakespeare on a bender." That is context. It shapes the statistical path the model takes to generate your response. As you chat, every previous message becomes part of that context. Over time, this accumulates and fills up the window. For most casual use, you will never hit this limit. But if you are working with a large codebase and pasting in massive amounts of file data, you might see the model say it is "compacting" the conversation. It is trying to compress your history so you can fit more new information in.

If your responses start feeling stale or disconnected, try starting a new session. A fresh context window gives the model a clean slate without the baggage of your previous, unrelated queries. It is a simple fix that often restores clarity.