gd-linear-convergence-conditions

IN premise — entries/2026/06/21/wiki-Gradient_descent-chunk-5.md

Created 2026-06-21T09:55:50+00:00

Gradient descent with fixed step size achieves linear convergence (error decreases by a constant factor each iteration) when the objective is both strongly convex and has Lipschitz continuous gradients

Dependents

These beliefs depend on this one:

• IN gradient-descent-conditionally-reliable — Gradient descent convergence is conditionally reliable — linear convergence is achievable under strong convexity and Lipschitz smoothness, but non-convex landscapes present saddle points that can trap or slow optimization, with momentum methods providing partial mitigation by accumulating velocity across the loss surface.