gd-momentum-update-rule

IN premise — entries/2026/06/21/wiki-Gradient_descent-chunk-4.md

Created 2026-06-21T09:55:50+00:00

The heavy ball (momentum) update rule is x^(k+1) = x^(k) - η·∇f(x^(k)) + β·(x^(k) - x^(k-1)), adding a fraction of the previous step to dampen oscillation

Dependents

These beliefs depend on this one:

• IN gradient-descent-conditionally-reliable — Gradient descent convergence is conditionally reliable — linear convergence is achievable under strong convexity and Lipschitz smoothness, but non-convex landscapes present saddle points that can trap or slow optimization, with momentum methods providing partial mitigation by accumulating velocity across the loss surface.