Stacking into an MLP

Take the 4-neuron layer from last lesson and feed its output vector into one more neuron. Now you have a multi-layer perceptron (MLP): an input layer, one or more hidden layers, and an output layer — each layer's output becoming the next layer's input.

Every layer bends its input through a nonlinearity before passing it on. Stack enough of these bends and the network can carve out decision boundaries far more intricate than any single layer could — a straight line becomes a curve, a curve becomes a loop, a loop becomes something that can separate genuinely tangled data. You'll see this directly in a couple of lessons, once this network actually trains on some.

Step through the forward pass and watch the same input flow through the hidden layer's four neurons, then collapse into one final number at the output neuron. This is the exact same left-to-right flow as a transformer block from Zero to GPT — data goes in one end, gets progressively reshaped by each layer, and comes out the other end as something new.

The full 2 → 4 → 1 network, one forward pass:

This network can compute an output for any input — but right now its weights are arbitrary, so the output means nothing. Making it mean something requires a loss function to say what "right" looks like.