AI & Digital Tools

AI Tokens and Context Windows Explained: Why Chatbots Forget and What It Costs

You paste a long report into an AI assistant and it stops halfway. A conversation that was going well suddenly loses track of what you said an hour ago. An API bill arrives and it is three times what you guessed. All three have the same cause, and it is a unit most people never see: the token.

• read_time: 10 min
• level: beginner
• tool: token estimator

You do not need to be a developer to benefit from understanding this. If you use ChatGPT, Claude, Gemini or any similar assistant for writing, research or code, tokens decide how much you can paste in, how long a conversation stays coherent and, on paid plans or APIs, what you spend. This guide explains the idea without maths, then gives you a tool to estimate tokens and cost for your own text.

What a token actually is

A language model does not see letters or whole words. Before your text reaches the model, a piece of software called a tokenizer splits it into pieces from a fixed vocabulary. Each piece is a token, and each token is converted to a number. The model works entirely on those numbers.

Common short words are usually a single token. Longer or rarer words are split into several. Punctuation, spaces and line breaks count too.

The exact split depends on the model. Each provider uses its own tokenizer, so the same paragraph can be 100 tokens for one model and 115 for another. The examples above show the pattern, not exact counts.

The rule of thumb

For ordinary English prose, OpenAI's documentation gives a widely used approximation: one token is roughly four characters, or about three-quarters of a word. So 1,000 tokens is around 750 words, and a 1,500-word article is around 2,000 tokens.

What the context window is

The context window is the model's working space: the maximum number of tokens it can take into account in a single request. Think of it as a desk of fixed size. Everything the model needs to see has to fit on the desk at the same moment.

People often assume only their latest message counts. In reality the desk has to hold four things:

Item 02 is the one that surprises people. A model has no memory between requests. The app creates the feeling of a continuous conversation by resending the whole transcript every time you press Enter. That is why the tenth message in a chat costs far more tokens than the first, even if you only typed one line.

Context window sizes differ a great deal between models and change with new releases, from a few thousand tokens in older or small local models to hundreds of thousands or more in current large ones. Check the provider's model documentation for the figure that applies to you; do not rely on a number from an old article.

Why chatbots forget

When a conversation grows past the window, something has to go. Different products handle this in different ways:

• Truncation. The oldest messages are dropped. The model simply never sees them again.
• Summarising. Earlier turns are compressed into a short summary. Details, exact wording and numbers are often lost in the process.
• Refusing. Some tools stop and tell you the conversation or file is too long.

There is also a softer effect well before the hard limit. Research on long contexts has found that models tend to use information at the very start and very end of a long context more reliably than material buried in the middle. A fact on page 40 of a 90-page paste is more likely to be overlooked than the same fact in a two-page paste.

So "the AI forgot my instructions" usually means one of three things: the instruction was trimmed out, it was flattened into a vague summary, or it is still present but lost among tens of thousands of other tokens.

Token and cost estimator

Paste any text to see roughly how many tokens it uses and how much of a context window it fills. If you use a paid API, enter the prices from your provider's pricing page to estimate cost. The tool works as you type.

How token pricing works

If you only use a chat app on a free or flat monthly plan, you are not billed per token. Tokens still matter, because they sit behind the message limits and "conversation too long" notices you run into. If you use an API, or a tool that passes API costs on to you, tokens are the bill.

Four things are true across the major providers at the time of writing. Confirm the details on the official pricing page, since they change:

• Prices are quoted per million tokens.
• Input and output are priced separately, and output usually costs several times more than input.
• Bigger models cost more than smaller ones, often by a wide margin, for the same text.
• Some providers discount repeated input (often called prompt caching) and non-urgent batch jobs.

Three worked examples

These use the four-characters rule and round numbers so you can follow the arithmetic. The prices are placeholders chosen to make the sums easy. They are not any provider's real rates.

Example 1: a single question

You type a 40-word question and get a 300-word answer. That is about 55 input tokens and 400 output tokens. Even on an expensive model this is a tiny fraction of a cent. One-off questions are never the cost problem.

Example 2: a long chat about one document

You paste a 6,000-word report (about 8,000 tokens) and ask 15 follow-up questions. Because the history is resent each turn, the report is processed 15 times: 120,000 input tokens for the document alone, before counting the growing transcript. At a placeholder price of 3 per million input tokens, that is roughly 0.36 just for re-reading the same report. Not ruinous, but 15 times what most people would guess.

Example 3: an automated job

A script summarises 2,000 product reviews, each about 150 words, with a 100-word instruction and a 60-word summary returned. Per review: about 335 input tokens and 80 output tokens. Across 2,000 reviews that is 670,000 input and 160,000 output tokens. At placeholder prices of 3 and 15 per million, the job costs about 2.01 plus 2.40, so 4.41 in total. Shortening the instruction from 100 words to 40 would cut input tokens by nearly a quarter.

The lesson from all three: volume and repetition drive cost, not any single message.

How to use fewer tokens without worse answers

1. Send only the relevant part. If your question is about section 4 of a contract, paste section 4 and the definitions it relies on, not the whole document.
2. Put the instruction next to the material. State the task clearly at the start, and for long pastes repeat the key question at the end, where it is less likely to be overlooked.
3. Ask for the length you need. "Answer in five bullet points" or "under 150 words" reduces output tokens, the expensive side, and usually gives a more useful answer.
4. Restart long threads. Carry forward a short summary of decisions instead of the full history.
5. Match the model to the job. Reformatting, classification and simple extraction rarely need the largest model.
6. Clean the input. Strip navigation menus, repeated headers, signatures and boilerplate from copied web pages and email threads before pasting.
7. Reuse a stable prefix. If you call an API with the same long instructions each time, keep that block identical and at the start so provider-side caching can apply where it is offered.

Shorter prompts are not automatically better. Cutting the context a model needs to do the job well produces worse answers and more retries, which costs more in the end. The aim is to remove what is irrelevant, not what is useful. A clear brief helps here; our content brief template shows how to say what matters in a few lines.

Common mistakes

• Assuming the model remembers earlier chats. Unless the product has a memory feature switched on, a new conversation starts empty.
• Treating a bigger window as better recall. A large window lets you paste more. It does not guarantee every detail in it is used.
• Estimating cost from input only. Output is usually the pricier side and is easy to forget.
• Trusting a word counter for non-English text or code. Both tokenise more heavily than English prose.
• Uploading files without checking what gets extracted. A PDF with scanned images, tables or hidden text can turn into far more, or far fewer, tokens than its page count suggests.
• Assuming the answer is complete. If a reply stops mid-sentence, it probably hit the output limit. Ask it to continue, and check nothing was skipped. For anything you publish, verify the content as described in how to fact-check AI content.

Before-you-send checklist

Run through this before a large paste or an automated job. Tick the items as you go.

FAQ

Conclusion

Tokens are the unit behind nearly every limit you meet when using AI tools: how much you can paste, how long a chat stays coherent, and what a paid request costs. The context window is a fixed desk that must hold your instructions, the history, new input and the reply together. Once you picture it that way, the fixes are obvious: send what is relevant, ask for the length you need, restart long threads with a short brief, and estimate before you run anything at volume.

Official references

• OpenAI Help: what are tokens and how to count them
• OpenAI tokenizer tool
• Claude documentation: context windows
• Google AI for Developers: understand and count tokens