The prediction machine

Assemble the pieces. A model built from stacked units (chapter 3), its knobs set by walking a loss downhill (chapters 1 and 2), reading text as tokens placed in a learned space (chapter 4). Point it at a mountain of text with one task: given the pieces so far, predict the next one. Cover a word, guess it, uncover, adjust the knobs, on and on across more text than a person could read in thousands of lifetimes.

Now watch what the trained machine does with your question. It does not produce an answer. It produces a scoresheet: for every piece in its vocabulary – tens of thousands of them – a score saying how strongly that piece fits as the next one. One piece gets picked from the sheet. It is glued onto the end of the text, and the lengthened text goes back in, and a fresh scoresheet comes out. Around and around. Every chatbot reply you have ever read was produced this way: one piece at a time, each drawn from a scoresheet computed from everything so far.

That is the loop. The rest of this chapter is about the two places the simple description hides something – how “one piece gets picked”, and what this loop does and does not tell you about the thing you are talking to.

The obvious rule – always take the top-scoring piece – turns out to write bad text. Flat, repetitive, oddly dead text that circles back on its own phrases; the reasons are half understood at best, but the effect is easy to reproduce and unmistakably worse. So instead, the choice is a weighted draw: high-scoring pieces are likely, low-scoring ones unlikely, and chance settles it.

How much chance is a dial, and the dial has a name you may have met: temperature. Turn it low and the draw sharpens toward the top of the scoresheet – cautious, predictable, close to that dead deterministic text. Turn it high and the odds flatten, low-ranked pieces start winning, and the text turns adventurous on its way to unhinged. There is no correct setting; there is a trade, and it sits in the open like a tone control.

The dial also dissolves a small everyday mystery: ask the same question twice, get two different answers. No mood swung, and the loop itself carries nothing from one conversation into the next. What changed is only the draw. Two runs, two sequences of weighted coin-flips, two paths through the space of plausible continuations – from the identical scoresheet both times.

“It is just predicting the next word.” You now know exactly how true that is – you have seen the whole loop, and there is nothing else in it. The sentence is regularly deployed to mean the behaviour is therefore trivial, a parlour trick inflated by marketing. That conclusion does not follow, and it is worth being precise about why.

Predicting the next piece well – across contracts, jokes, code, condolence letters, chess commentary – is not a shallow task. The training text was written by people doing all the things people do: arguing, planning, calculating, keeping characters straight across a plot. A machine cannot score well on the next word of a detective novel’s final page without something inside it that functions like tracking who did it. “Just prediction” names the task. It puts no ceiling on the structure the task forces into the knobs – and driving the loss down on that task built real, intricate structure. Dismissing it is not scepticism; it is a different way of not looking.

The reverse error is just as common and just as cheap: reading the fluency as understanding, intent, a someone. Fluency is precisely what the training target rewards – text shaped like text people write, feelings included, in the exact costume-sense of chapter 0. Of all the evidence you could have about whether anyone is home, persuasive text is the piece this machine was specifically optimised to produce. It is the one signal you should trust least.

So hold both, because both are true at once: nothing but prediction, and prediction is not nothing. Where between those poles the reality sits – what the learned structure amounts to, whether functions-like-tracking is meaningfully different from tracking – is not settled. The field measures, probes, publishes and disagrees, and this primer will not pretend otherwise in either direction. Anyone who tells you the question is closed is selling something; the honest position is a working picture of the mechanism and an open verdict on what it adds up to. What the mechanism cannot do, though, follows from its shape regardless – and that is the last chapter.