AI Context Window Explained: What It Is and Why It Fills
An AI’s context window is the amount of text the model can look at while it writes one reply: your messages, its earlier answers, attached files and the reply itself. Anything outside the window, the model can’t see. That is why long chats forget their own beginning.
Think of it as working memory for this one conversation. General knowledge came from training; the window is what the model holds in front of it right now.
What is a context window in AI?
Anthropic’s documentation defines the context window as “all the text a language model can reference when generating a response, including the response itself”, and contrasts it with the training data. The same page calls it a “working memory” for the model.
Every time you send a message, the app sends the model the whole conversation so far, plus your new message. The model reads all of it, writes a reply, and that reply joins the pile for the next turn. The pile grows with every exchange until it hits the limit.
What counts toward the context window?
More than you might expect. Per Anthropic’s docs, everything in the request counts: the system prompt, every message including attached files and tool results, the tool definitions, plus the output the model writes, including its reasoning.
Here is what typically fills a chat’s window, from a user’s point of view:
| What you see | Does it count? | Notes |
|---|---|---|
| Your messages | Yes | Every one, from the start of the chat |
| The AI’s replies | Yes | Long answers fill the window fast |
| Uploaded files and pasted text | Yes | A long PDF can take a big share |
| Custom instructions or project instructions | Yes | Sent with every message |
| Tool and connector results | Yes | Search results and web pages add up |
| Saved memories the app injects | Yes, when included | Apps add selected memories to the prompt |
| Your other chats | No | Unless the app pulls them in on purpose |
The last row is the important one. A new chat starts with an empty window apart from instructions and whatever memory the app chooses to add.
How big is an AI context window?
It depends on the model, and the numbers change often, so check the vendor’s current page before you rely on one. Anthropic’s docs, for example, list a 1M-token window for its current models and 200k for some older ones. OpenAI and Google publish their limits per model on their own model pages, and a chat app may not give you a model’s full window, so check the app’s plan page too.
A token is a chunk of text, often part of a word. OpenAI’s rule of thumb is that one token is roughly four characters of common English text. So a window measured in hundreds of thousands of tokens holds a lot: several long documents and a long conversation at once.
Why does the AI forget things in a long chat?
Two different reasons, and they need different fixes.
The window is full. Once a conversation passes the limit, something has to go. Anthropic notes that chat interfaces such as claude.ai can manage the window on a rolling “first in, first out” basis, which means the oldest turns drop out. Another approach is compaction, which summarizes earlier parts of the conversation so it can keep going. Either way, details from the start are gone or compressed.
The window is crowded. Even before it’s full, more text is not always better. Anthropic’s docs say that as token count grows, accuracy and recall degrade, which they call context rot. A fact buried deep in a long chat is easier to miss than the same fact in a short, focused prompt.
So a bigger window helps, but it doesn’t remove the need to keep context tidy.
What is the difference between a context window and AI memory?
These two get mixed up all the time, and that causes most of the confusion about why an AI “forgot.”
| Context window | Memory features | External notes | |
|---|---|---|---|
| What it is | Text the model sees right now | Facts the app saves and adds to future chats | Documents you keep and the AI reads on demand |
| Lasts | One conversation | Across chats in one app | As long as you keep them |
| You control it | Partly, by what you paste | Partly, by reviewing saved items | Fully |
| Moves between apps | No | No | Yes, if the AI can reach them |
Memory features, like ChatGPT’s saved memories and chat history reference, don’t make the window bigger. What they change is what goes into it: a few stored facts or details from past chats, included with your new message. For how the two biggest apps handle this, see Claude memory vs ChatGPT memory. If you’ve hit the limit on stored memories, ChatGPT memory full covers what to do.
How do you work within the context window?
A few habits make a bigger difference than a bigger model:
- Start a new chat per task. When the topic changes, a fresh window beats dragging the old conversation along.
- Summarize before you switch. Ask the AI for a short summary of decisions so far, then load that summary into the next chat.
- Keep instructions short. Project or custom instructions ride along with every message. Every line in them should earn its place.
- Load documents on demand. Instead of attaching every file up front, keep them somewhere the AI can search, and let it read only what the task needs.
- Write decisions down outside the chat. The chat will end. A note won’t.
The same applies in coding tools, where rules files load into every session. See Cursor memory between sessions for that version of the problem.
Can you share context between different AI apps?
Not through the context window itself. Each app builds its own window from its own chat. To carry context from Claude to ChatGPT, or into Cursor, you need a source they can all read, like a notes server connected over MCP. That is the approach in sharing memory between Claude and ChatGPT and an MCP server for notes.
FAQ
Is a bigger context window always better?
No. It lets you include more, but recall can drop as the window fills. A focused prompt with the right three documents often beats a huge one with everything.
Does the context window reset in a new chat?
Yes. A new chat starts empty, apart from instructions and any memory or project files the app adds automatically.
Are tokens the same as words?
No. A token is often part of a word. For English text, a common estimate is about four characters per token, so a token count runs higher than a word count.
Do uploaded files count toward the context window?
Yes, when their text is sent to the model. Some apps avoid sending everything: Claude Projects, for example, can switch to a retrieval mode on paid plans when project knowledge gets close to the limit.
Where Valta fits
Valta is a place to keep the context that shouldn’t live in one chat. It’s a hosted vault of Markdown notes with an MCP server. Any AI that supports MCP can search it and read only the notes a task needs, so you stop pasting everything into the window. When a chat produces a decision worth keeping, your AI saves it as a note, and Valta returns related notes so the AI can link them. Valta runs no model of its own; the window and the writing stay with your AI.
You can try it with one vault and 100 notes on the Free plan. Create an account.