Vectors and similarity

The primer left you with a picture worth taking seriously: every word a machine reads becomes a list of numbers, and lists that behave alike sit near each other. This chapter is about earning that picture – because “lists of numbers” and “near each other” belong to two different worlds, and the bridge between them is the first genuinely load-bearing piece of maths in machine learning.

Start with the list itself. A weather report is one: temperature 19, wind 12, humidity 68. Three numbers, in a fixed order, describing one thing. A character sheet in a game is one. So is a row in a spreadsheet. A machine-learning model deals in exactly these – fixed-length lists where position carries meaning – and the field’s word for one is a vector.

Two numbers pick out a spot on a map: this far east, this far north. That is the whole idea, and it does not stop at two. Three numbers pick out a spot in a room. The weather report’s three numbers pick out a spot in a “weather space” whose directions are temperature, wind and humidity. A mild, still, damp day sits in one corner of that space; a hot, windy, dry one sits far away in another.

Nobody can picture the version with three hundred directions, and nobody needs to. The move that matters is smaller and stranger: treating a description as a location. Once every thing is a place, questions about things become questions about geography. Which days felt alike? The ones whose spots are close. What does this direction mean? Follow it and see what changes. The rest of the chapter is just learning to measure in that country.

On a map, distance is a tape measure between two pins. In weather space it works the same way, with one honest wrinkle: every direction of the description contributes. Two days that differ a little in temperature, a little in wind and a little in humidity are further apart than two days that differ only in wind – all the little differences pool into one overall gap.

That pooled gap is a single number, and it is the machine’s entire notion of “these two things resemble each other”. No understanding, no squint of judgement – a measurement. When the primer said similar words end up near each other, this is the ruler that sentence was leaning on. What is genuinely remarkable, and worth pausing on, is that a ruler this dumb ends up tracking resemblance at all. It does so only because training put resembling things near each other in the first place – the geography was learned, and the ruler just reads it.

Distance asks “how far apart?” There is a second question, subtly different and more useful more often: “how much do these two point the same way?” Two long arrows can be far apart at their tips yet aimed identically; a short arrow and a long one can aim the same way despite their different reach. Pointing-the-same-way is its own kind of likeness – agreement of character rather than coincidence of size.

The machine that measures it is called the dot product, and it works like a compatibility score. Where both descriptions are high, it adds a lot; where both are low, a little; where one is high and the other low, it subtracts. Same character everywhere: big positive score. Opposite characters: negative. Nothing in common: near zero. One multiply-and-add per direction, and out comes a single number that behaves uncannily like “how alike are these?”

That number is the payoff the chapter was named for. When a model decides which earlier words matter for the word it is about to produce, the deciding arithmetic is this score, computed between vectors. You now know the operation that sentence runs on.