TALAMANA · THE AI LITERACY MAP FOR ARCHITECTURE AND DESIGN · Generative Mechanics · AGE 16—18 · FACTUAL · DURABLE
Sampling: from a spread to a token
The model produces a spread of probabilities, and a decoding rule picks one token.
The idea
At each step the model does not produce a word. It produces a spread: a probability for every token that could come next. Something has to pick one — that step is sampling, or decoding. Greedy decoding takes the most likely token every time. Sampling draws from the spread, weighted by probability. Temperature reshapes the spread before the draw — low sharpens it towards the likely tokens, high flattens it so rarer ones get a chance. Top-p trims it to the smallest set whose probabilities add up to p. The picked token joins the text; the next spread follows.
Why it matters
Temperature, top-p and seed are not magic dials. Each one changes how a spread becomes a token. Once you see the step, you can predict what a setting will do before you touch it.
See it in the studio
You ask for ten names for a courtyard housing scheme at a low temperature and get ten that sound the same. You raise it and get stranger, looser names, and one or two that make no sense. Neither run was "the model's opinion". Both were draws from the same spread, picked by a different rule.
Watch for this
Calling a low-temperature answer "the correct one". Greedy or near-greedy decoding gives you the most probable continuation, token by token. Probable is not true, and the most probable path is also the one most likely to repeat itself in a long answer.
Try it
Find a tool that exposes temperature, such as an API playground. Ask for the next sentence of a brief you wrote, five times at temperature 0, then five times at a high temperature. At 0 the answers will be nearly identical on the same setup; high up they scatter. Then set a fixed seed at the high temperature and run twice. On the same setup you should get the same pick, twice. You have just separated the spread from the rule from the dice.
Prove it
Explain the step between "the model has a spread of probabilities" and "a word appears on screen", and say what greedy decoding, temperature and top-p each do to that step.
How it works
The model's last layer gives a number for every token in its vocabulary — often tens of thousands or more. A function called softmax turns those numbers into probabilities that add to one. That is the spread. Temperature divides the numbers before softmax: below one, the gaps between tokens widen, so the top token dominates; above one, the gaps shrink. Top-k keeps the k most likely tokens; top-p keeps the smallest set whose probabilities sum past p, which adapts to how confident the spread is. A random number then picks within what is left, and the seed is the start value of that random number generator — which is why SEED on this strand can repeat a run on the same setup and why THE DRAW on Foundations says every output is one draw. Hugging Face's own documentation names greedy search as its default and describes sampling, temperature, top-k and top-p as the alternatives. Image generators have their own version of this: the seed fixes the starting noise and the sampler schedules the steps, DIFFUSION on this strand explains how. Products usually hide the settings and choose them for you; what they chose is part of the product's behaviour, not the model's. KINDS OF DOUBT on this strand takes the probability in the spread further: it is a number about the next token, not a statement about the world.
What this idea builds on
What this idea opens up
Sources
Open this idea on the map · The complete map · Logika · RBDS AI Lab, India · revised every edition.