Matrices as maps
The primer said a neuron is a weighted sum and a network is neurons stacked. Both sentences compress into one object: the matrix. Read as a grid of numbers it is bookkeeping; read as a machine that moves space it explains what a layer does, and why depth without a twist between the layers collapses into no depth at all.
Chapter 1 turned descriptions into places. This chapter is about the machines that move them.
Keep the weather picture. Suppose you want tomorrow’s forecast from today’s report: some recipe that takes in (temperature, wind, humidity) and puts out new numbers. The simplest recipe with any teeth is the one the primer met inside a neuron – for each output, take a weighted mix of every input. Tomorrow’s temperature might be most of today’s, minus a little for wind, plus a touch for humidity. Write down one mixing rule per output and you have a grid of knobs: one row per output, one knob per input.
That grid is a , and the whole chapter turns on a change of posture toward it. Stop reading it as a table to look numbers up in. Start reading it as a machine: places go in, places come out, and the grid is the machine’s gearing.
2.1A machine that moves space
Here is the picture to keep. Lay a sheet of graph paper over the plane and let a matrix act on every point at once. The sheet stretches, squashes, rotates, shears, maybe flips – but it does so evenly. Straight lines stay straight. The centre stays put. Evenly-spaced gridlines stay evenly spaced. A matrix redraws the whole country with one consistent gesture; what it cannot do is treat one neighbourhood differently from another, bend a line, or move the origin.
This even-handedness is a real limitation, and you have met its consequence before. The primer’s single neuron could only draw a straight dividing line – that is the even gesture, seen from the decision-boundary side. Hold onto the limitation; §2.2 turns it into the chapter’s main event.
2.2Depth, and the collapse
The primer’s answer to the straight-line limit was stacking: feed one layer’s output to the next, let simple machines compound into a clever one. Here is the uncomfortable question this chapter exists to ask: does stacking these even-handed machines actually buy anything?
Run the graph paper through machine A, then through machine B. Stretch, then rotate. The result is – another even gesture. A third machine C exists that does in one step exactly what A-then-B does. And this does not stop at two: stack a hundred matrix machines and the whole tower is exactly equivalent to a single matrix machine. All that apparent depth, and the family of possible behaviours has not grown at all. Straight lines in, straight lines out, one even gesture, however tall the stack.
So the primer’s story was incomplete in an important way. Stacking alone buys nothing. What buys everything is the small twist the next section adds between the layers – and now you know why it has to be there.
2.3What a layer is
Assemble the pieces and the mystery object of the primer has a full description in one breath: a layer is an even-handed remapping of space (the grid of knobs), a shift (the bias), and a twist (the switch). Mix, move, bend. A network is that, repeated – and because the twist breaks the collapse, each repetition genuinely adds reach.
There is a live demonstration of the difference an honest twist makes. The spiral dataset below cannot be cut by any straight line, and no stack of untwisted machines could ever separate it – you now know the proof, not just the slogan. Watch a small twisted stack learn it in seconds.
The tool is the destination – the prose stops here and you go and break it yourself.