What Are LLM Tokens, and Why Does AI Charge by Them?

Almost everyone pauses at the same line of their first AI service bill. The unit of charge is not requests, not words, not minutes, but an unfamiliar one — the token. So many in, so many out, total due.

To read that bill, accept one premise first: a large language model does not see text the way you do. You see a sentence. It sees a row of building blocks.

The model reads blocks, not words

Before your text reaches the model, a slicing step cuts it up, and every resulting piece is a token. Common words usually survive as one prefabricated block; rare words get split into smaller pieces and reassembled; punctuation and spaces occupy blocks of their own. The model reads block by block and writes the same way — it does not compose a full sentence in its head and then speak; it lays down one block at a time, glancing back at all the blocks so far before placing the next.

So a token is neither a character nor a word. It is the smallest unit in which a model handles text, and the mould that cuts the blocks belongs to each model's own tokeniser — not to any dictionary.

The same meaning can cost a different number of blocks

That mould produces an unintuitive result: the same passage, written in a different language or fed to a different model, can come out as a noticeably different token count — like brick sets from different makers, where the same house takes a different number of pieces because the moulds differ. There is no universal exchange rate, and any "one word equals so many tokens" shortcut expires the moment you switch models. What is worth keeping is a sense of scale: a short message lives in the tens to hundreds of tokens; a long manual runs to the tens or hundreds of thousands. When a precise number matters, mainstream model services provide token counters — have a technical colleague run one.

Why bill by the block: every block passes through the machine

Per-token pricing is not a pricing trick; it is the actual shape of the cost. Every block the model handles requires real computation — each block read in, and each block written out. Hence the two columns on the bill: input and output are billed separately. And input is larger than you think: besides the sentence you typed, it carries the system prompt (the platform's preset role and rules), whatever material you pasted, and the conversation history — which is resent with every turn, so the longer the conversation drags on, the thicker that part grows. As an aside, most services price output blocks higher than input blocks — generating costs more compute than reading — so getting the model to answer concisely and precisely is not just a matter of style; it is real money.

This also explains why longer context costs more: there are simply more blocks, and each new block the model lays down requires another look across everything already on the table — the fuller the table, the harder each step works. What context actually is, and why it also "forgets", is covered separately in Context Windows: Why AI Forgets Things Mid-Conversation.

A rough bill for one business scenario

For a business, tokens matter as budget, not trivia. A rough monthly figure for one scenario takes three numbers: how large a typical call's input is (prompt plus material plus history), how large the output is, and how many calls per month. Multiply the three, then by the unit price, and the order of magnitude appears. Customer-service Q&A is a good example: each call carries a few passages of knowledge-base material, so input is sizeable; answers are short, so output is small; but hundreds of calls a day push the total up. The estimate need not be precise — it needs to exist before the project is approved. Add a monthly usage cap and an overage alert, and the budget goes from estimated to governed.

The same arithmetic shows why one popular habit is a bad deal: stuffing the entire product manual into every conversation. A manual is hundreds of thousands of blocks, billed in full on every single question — expensive, and self-defeating too, because the longer the material, the more easily the model overlooks the key passage. The saner route is retrieval: fetch only the few passages relevant to this question and hand the model those — which is exactly what RAG does.

Two misunderstandings worth clearing up

One is treating tokens as a word count. Between a contract that says "handles so many thousand words" and a bill settled in tokens sit differences of tokeniser, language and format — worth having a technical colleague reconcile before signing. The other is assuming a cheap unit price means a cheap total. Total cost is unit price times volume times rework: a cheaper model that understands less may need longer prompts, more correction rounds and more human touch-up, and the total may come out worse. Nor is switching models the only lever — trimming prompts, caching repeated content, and matching model tiers to task difficulty all have room. Controlling Enterprise AI Costs goes further.

Two sentences are enough for a decision-maker

First: the token is the utility meter of the AI era — you never have to read it yourself, but you should know which behaviours burn through it: long material, long conversations, high frequency. Second: at approval time, ask one question — "roughly how many tokens a month will this scenario use, and who is watching that number?" If someone can answer, the month-end bill will never be how you find out.