Day 25: Whiteboard session: chain rule through a 2-layer net
Consolidation: make it interview-ready
The roadmap's Stage 1 exit criterion is explicit: *'micrograd-style backprop from scratch; whiteboard the chain rule through a 2-layer net.'* Today isn't new material — it's turning what you built into something you can explain cold, on a whiteboard, without notes. That translation from 'I coded it' to 'I can teach it' is what separates a portfolio project from an interview win.
The narrative to rehearse
- Forward pass: inputs flow through layer 1 (
h = tanh(W₁x + b₁)), then layer 2 (ŷ = W₂h + b₂), then the lossL = (ŷ − y)². Draw the graph left to right. - Seed: set
dL/dL = 1at the output. - Backward through the loss:
dL/dŷ = 2(ŷ − y). - Backward through layer 2:
dL/dW₂ = dL/dŷ · h,dL/db₂ = dL/dŷ, anddL/dh = dL/dŷ · W₂. - Backward through the tanh and layer 1: multiply by
tanh'=(1 − h²), thendL/dW₁ = dL/d(pre-activation) · x. - Update: every parameter steps against its gradient. Emphasize the gradients *multiply* along the chain — that's the chain rule made visible.
The one sentence that proves you get it
"Backprop is just the chain rule applied mechanically backward through the computation graph, multiplying each operation's local derivative to find how every parameter affects the final loss." Say that, then draw the graph. If you can, you've cleared the hardest conceptual bar in the entire roadmap.
Exit-criterion rehearsal — do these without notes
When backpropagating through a tanh activation h = tanh(z), what factor do you multiply the incoming gradient by?
