#Matrix calculus

2026-09-16·—READS·—LIKES·Learning AI with AI

Maths Catch-Up · Summation Is What Reuse Costs: sum(0) or sum(1), Just Compare the Indices

Part 3 gave the Σ of the multivariable chain rule and part 13 used it to derive the linear layer — but that was one special case. This part turns it into a general method: the addition comes from the total differential, the four shapes differ only in which entries of one Jacobian are zero, and whether to sum or broadcast — and along which axis — reads straight off the two index sets.

2026-09-07·—READS·—LIKES·Learning AI with AI

Maths Catch-Up · The Derivative of a Copy Is 1: How Gradient Reaches the Embedding Table

A gradient always has the shape of whatever it differentiates, and two rules are enough to derive the backward formulas of a linear layer. The Jacobian of C[X] holds only 0s and 1s — and both properties of the embedding gradient, sparsity and accumulation, are written on that one table.