2 inputs → 2 hidden → 1 output ｜ η = 1

The same cat, three times around

Every round is “forward for the cost → backward for the gradients → update the parameters”. Click any number in the diagram to see how it was worked out.

z = w·a_prev + b
Weighted sum: this layer’s parameters × the previous layer’s output, plus a bias
σ(z) = 1/(1+e⁻ᶻ)
Activation: squashes anything into 0–1; the result is called a
C = (a2 − y)²
Cost: the gap between output and target, squared
σ′(z) = a(1−a)
Slope of the activation (this is how σ′ is computed, not an extra factor)
δ = ∂C/∂z
How sensitive the cost is to this layer’s weighted sum
w ← w − η·∂C/∂w
Update rule: take one step down the gradient
Forward

—

—

Three training rounds through a two-layer network w1_11 w1_12 w1_21 w1_22 w2_1 w2_2 x1 ears 0.9 x2 eyes 0.8 h1 h2 Output C target y = 1.0 ↑ tap any number to see how it was worked out

Use ← → to step through. Green = forward, orange = backward, purple = parameter update. After three rounds the cost falls from 0.1498 to 0.0495.