backprop-weight-gradient-outer-product

IN premise — entries/2026/06/21/wiki-Backpropagation-chunk-2.md

Created 2026-06-21T09:55:49+00:00

The weight gradient at layer l is computed as an outer product: ∇_{W^l} C = δ^l · (a^(l-1))^T, which is why activations from the forward pass must be cached.

Dependents

These beliefs depend on this one:

• IN backprop-efficient-recursive-gradient-factorization — Backpropagation's computational efficiency stems from a recursive factored structure: the error signal delta propagates backward layer-by-layer via matrix-vector products, each weight gradient factors into a compact o_i * delta_j form, and the full weight gradient is a simple outer product of local quantities — avoiding redundant computation entirely.