How LLMs Actually Work: Tokens, Context and Sampling

I apply these concepts. I hope you enjoy this new series as much as I'm enjoying writing it. What is an LLM? How does it learn?

Tokenization Context window The Transformer Sampling What this means when you call the API LLM means ✨Large Language Model✨ What does each letter in LLM mean? Large: it's huge. Size is measured in parameters. For example, DeepSeek-R1, an open model, has 671 billion parameters.

Language: it works with language: text in, text out. Model: it's a mathematical model, specifically a neural network with an architecture called the Transformer. It's different from a conventional system (a computer one) because it doesn't have written rules. There's no if (question === "...") anywhere, it learned by reading a large amount of text Pre-training.

During training, the model reads a massive amount of text and "plays a game" where it tries to predict the next token. Each time, it compares its guess with the real answer and measures how wrong it was. Then, an algorithm called backpropagation is used to measure how much each parameter contributed to the error, and those parameters are updated so the model gets a better answer next time. To explain it better, it's like having a panel with a lot of controls, where each combination of those controls gives us a different answer.

At the beginning, the answers are wrong, but every time we adjust the controls, they get a little closer to the right one. The algorithm helps us detect which controls we need to change to get a better answer. Then we repeat that over and over, billions of times, until the controls are set the right way. Post-training.

Once the first part of the training is done, the model knows a lot, but it only knows how to continue text, not how to answer you. So we keep training it, now with examples of "good" conversations and with human feedback: people read the model's answers and rate which ones are better. I'm sure you've seen those job posts where you "help" train AI models. That's exactly this part.

Here is a route of what happens to your message when you click on send: Your text (raw input) ↓ Tokenization (text to tokens) ↓ Context window (chat history with your message) ↓ Transformer ↓ Sampling (temperature, top p, top k) ↓ Output text Here is a nicer image generated by Claude for this purpose: This is the first step. Since the model doesn't read text, your text has to be transformed into tokens (a list of numbers) before being processed. That's tokenization. What is a token?

"Hello world" "Hello" → 15496 " world" → 995 Note: Those numbers are always the same for the same tokenizer, and they map each piece of text to a specific number. The context window is everything the model can "see" when it generates a response. It's its working memory, and it has a limit measured in tokens. When you call an API, the context is usually made of: System prompt: written by the developer.

It defines the model's behavior, role and restrictions. The end user of an app usually doesn't see it. Chat history: all the previous messages in the conversation, from both the user and the model. Current message: the new message, added at the end.

This is where the "thinking" happens. 1. Attention Take the word bank: "I sat on the bank of the river" → bank pays attention to "river" → 🏞️ "I deposited money in the bank" → bank pays attention to "money" → 🏦 Same token, different meaning, and attention is the mechanism that figures out which one it is. It means that the model assigns a weight to every other token: high weight for the ones that matter, low weight for the rest.

2. Feed-forward network The Transformer has processed all the tokens and now has a list of probabilities for what the next token could be: "The sky is..." blue → 74% red → 12% grey → 8% ... → 6% Sampling is the mechanism that takes this list and decides which token to use. Once a token is chosen, it's added to the context, and the whole process runs again to predict the next one.

The model doesn't write the answer all at once: it generates it token by token, which is why you see responses appear word by word in a chat. The loop stops when the model generates a special end token or when it reaches the maximum number of tokens allowed. Then the IDs are converted back to text (detokenization), and that's what you see on screen. Let's connect all of this with real code.

Here's a call using the official Anthropic SDK for TypeScript: import Anthropic from "@anthropic-ai/sdk"; const client = new Anthropic(); // reads ANTHROPIC_API_KEY from the environment const response = await client.messages.create({ model: "claude-sonnet-5-5", max_tokens: 300, // limit for the generated tokens (the loop stops here) system: "You are a senior backend engineer. Answer in 3 sentences max.", // system prompt messages: [ // chat history: you resend it on every call (the model is stateless) { role: "user", content: "What is a message queue?" }, { role: "assistant", content: "A message queue is a buffer that..." }, // current message { role: "user", content: "And when should I use one?" }, ], }); for (const block of response.content) { if (block.type === "text") console.log(block.text); } console.log(response.usage); // input and output tokens: what you pay for console.log(response.stop_reason); // why the loop stopped, e.g. "end_turn" or "max_tokens" Look at how every concept shows up: system and messages are your context window. max_tokens is the limit for the generation loop.

usage tells you how many tokens went in and out. stop_reason tells you whether the model finished on its own or hit the limit. Well, this is the first post about LLMs and how they work. In the next post, we'll keep working with the API: how to get structured outputs (JSON you can actually trust in your code) and how to give the model tools to call.

See you next week :D