backprop-backward-vs-forward-complexity

IN premise — entries/2026/06/21/wiki-Backpropagation-chunk-2.md

Created 2026-06-21T09:55:49+00:00

Backward-mode differentiation multiplies a vector by a matrix at each layer (O(n²) per layer), while forward-mode multiplies a matrix by a matrix (O(n³) per layer), making backward mode computationally superior for computing loss gradients.