encoder-decoder-rnn-precursor-to-transformer

IN premise — entries/2026/06/21/wiki-Recurrent_neural_network-chunk-2.md

Created 2026-06-21T09:55:53+00:00

Encoder-decoder RNNs were state-of-the-art for neural machine translation from 2014–2017 and were a direct precursor to the Transformer architecture.

Dependents

These beliefs depend on this one:

• IN attention-bridge-rnn-to-transformer — The attention mechanism served as the transitional innovation from RNN-based to Transformer-based architectures, first augmenting encoder-decoder RNNs to solve the fixed-size bottleneck, then replacing recurrence entirely.
• IN sequence-modeling-three-generations — Sequence modeling evolved through three generations — RNNs, LSTMs, Transformers — each solving a specific limitation of its predecessor (vanishing gradients for LSTMs, sequential bottleneck for Transformers).