lstm-solves-vanishing-gradient

IN premise — entries/2026/06/21/wiki-Recurrent_neural_network-chunk-1.md

Created 2026-06-21T09:55:52+00:00

LSTM (Long Short-Term Memory), invented by Hochreiter and Schmidhuber (1995/1997), uses gating mechanisms to preserve long-term dependencies and was specifically designed to address the vanishing gradient problem in vanilla RNNs.

Dependents

These beliefs depend on this one:

• IN residual-connections-and-lstm-gating-are-convergent-solutions — Residual connections (ResNet) and LSTM gating are convergent solutions to the same underlying problem — both enable gradient flow through deep computation graphs, with LSTM's forget gate solving vanishing gradients in recurrent (temporal) depth and skip connections solving the degradation problem in feedforward (spatial) depth — suggesting that identity-preserving shortcuts are a universal architectural principle for deep networks regardless of topology.