2 inputs → 2 hidden → 2 outputs ｜ 3 rounds ｜ η = 1

One hidden neuron, two downstream paths

The output layer now has two neurons (cat / dog). h1 feeds both of them, so on the way back the contributions of the two paths must be added together. Three rounds; click any number in the diagram to see the derivation.

z = w·a_prev + b
Weighted sum
a = σ(z) = 1/(1+e⁻ᶻ)
Activation, squashed into 0–1
C = Σ(a2 − y)²
Sum of both outputs’ squared errors
σ′(z) = a(1−a)
Slope of the activation
δ = ∂C/∂z
How much this neuron is to blame
∂C/∂a1 = Σⱼ δⱼ·wⱼ
Multi-path sum — the point of this demo
Forward

—

—

Multi-path backpropagation in a 2-2-2 network x1 ears 0.9 x2 eyes 0.8 h1 h2 Cat Dog C target y = (cat 1.0, dog 0.0) ↑ tap any number to see how it was worked out

Use ← → to step through. The two red steps are the multi-path sum — the link that does not exist in a single-output network.