Karpathy V1 · Neuron / Layer / MLP

One n(x) call, four classes passing the baton

MLP(3, [4,4,1]) with input [2.0, 3.0, -1.0]. Watch the call drill down layer by layer, then surface again.

MLP

—

MLP.__call__(x)

Feeds x to each layer in turn, overwriting x with the result

—

Layer.__call__(x)

Feeds the same x to every neuron here, collects the outputs

—

Neuron.__call__(x)

Weighted sum + bias, then through tanh

—

Value.__mul__ / __add__ / tanh

Each operation builds a graph node and hangs a _backward on it

—

Network · where the computation is now

CODE

Call stack

MLP.__call__

　└ Layer.__call__

　　└ Neuron.__call__

　　　└ Value.__mul__ …

Use ← → to step through. Purple = MLP, green = Layer, orange = Neuron, red = Value.