gd-saddle-point-limitation

IN premise — entries/2026/06/21/wiki-Gradient_descent-chunk-4.md

Created 2026-06-21T09:55:50+00:00

Gradient descent can slow down or get stuck near saddle points, which is a known limitation for non-convex optimization

Dependents

These beliefs depend on this one:

• IN gradient-descent-conditionally-reliable — Gradient descent convergence is conditionally reliable — linear convergence is achievable under strong convexity and Lipschitz smoothness, but non-convex landscapes present saddle points that can trap or slow optimization, with momentum methods providing partial mitigation by accumulating velocity across the loss surface.
• OUT gradient-descent-effective-global-optimizer — Gradient descent with backpropagation effectively finds global optima for neural network training in practice, despite lacking global guarantees.