Follow Hi 你! through position embeddings, LN and QKV, two residual additions, the MLP, vocabulary logits, cross-entropy and AdamW, then trace fixed-parameter generation and incremental decoding. Three diagrams and two interactive walkthroughs distinguish parameters, activations, gradients and context.
The previous three parts derived all twenty-six steps from the loss to every parameter, but that was opening the black box and walking through it. This one puts the box back together: cross-entropy's eight steps compressed into one line, BatchNorm's ten into one, both by the same method — count how many times a variable appears in the forward pass and the backward has that many terms. Then backward() is commented out and the model trains for 200,000 steps: val loss 2.1105, against PyTorch's 2.1106.
The previous part finished deriving the test for tensor calculus; this one applies it eight times over, working back from the loss to the logits. What is worth taking away is not the eight expressions but a way of marking your own work, three operating rules, and one property — a gradient that must be exactly zero — that doubles as a debugging tool.
The hidden layer is chapter 1's weighted sum, computed for 228,146 samples at once. A tensor is one strip of memory plus a manual: view costs nothing, a transposed tensor cannot be viewed — and hand-written loss matches F.cross_entropy until it returns nan.
Cat-versus-dog and predicting the next character are the same problem, differing only in that the class count goes from 2 to 27. The three parts that difference requires: one-hot, softmax, cross-entropy — and why cross-entropy is not a fancier squared error.