Probability and expectation

Chapter 4 ended on a design constraint: for the walk to work, wrongness has to respond smoothly to the knobs. A verdict cannot do that. If the model’s answer is flatly “cat”, a small nudge of any knob usually changes nothing – then suddenly flips the verdict. All cliff, no slope; the walker feels nothing until everything changes at once.

The fix is one you have already seen in action. The primer’s machine never said “the next word is sat” – it gave every possible next word a share of its belief. Shares of belief move smoothly when knobs move: a nudge shifts belief a little toward or away from the truth, and the walker feels it. Answering with a spread instead of a verdict is not hedging; it is what makes the machine trainable at all.

This chapter makes the spread respectable: what counts as one, how a machine’s raw scores become one, and what you can do with one once you have it.

Inside the machine, the last layer produces raw scores – one per option, any size, positive or negative, not remotely adding to one. Turning scores into a legal spread of belief takes a fixed two-step recipe. First, make everything positive in a way that preserves order and exaggerates gaps – a score two points higher becomes several times larger, not two units larger. Then divide each by the total, so the shares sum to one.

The recipe’s name is softmax, and the name is apt: a hard max would hand everything to the winner; softmax hands the winner the biggest share and leaves honest slivers on the runners-up. Those slivers are where all the interesting behaviour lives – they are what the machine “also considered”, and what sampling (§5.3) occasionally promotes.

One more instrument and the kit is complete. When outcomes are spread over possibilities, single questions still need single answers: how wrong is the model typically? What result should we expect? The instrument is the probability-weighted average: value of each outcome, times its share of belief, summed. Not the outcome you will get – often not even a possible outcome (no family has 1.9 children) – but the number the results average toward over many draws.

It matters here for one structural reason: training’s wrongness number has always been an average. The primer averaged misses over a pile of examples; that pile is itself a sample from the world’s spread of possible examples. Training is, in truth, minimising an expected wrongness – the average over everything the world could throw at the model, estimated by the pile at hand. Hold that thought; it is the precise seam where this chapter and the next join, and where the overfitting story from the primer becomes a theorem-shaped statement.

A spread of belief still has to become one word on the screen. Always picking the biggest share reads as flat and looping – the primer said so, and now you can see the mechanism: the runner-up slivers never get their turn, so the machine’s second thoughts simply vanish. Instead the choice is sampled: belief becomes odds, dice are rolled, and a 3% sliver wins 3% of the time.

The temperature dial from the primer also has a mechanism now. It reshapes the spread before the roll. Turned down, gaps between scores count for more – belief piles onto the favourite, and at the extreme the dice become a formality. Turned up, gaps count for less – belief flattens toward equal shares, slivers grow, and the machine’s choices loosen. Nothing about the scores changes; only how sternly gaps are converted into odds.