transformer-no-recurrence-enables-parallelism

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-1.md

Created 2026-06-21T09:55:54+00:00

Transformers eliminate sequential recurrence found in RNNs/LSTMs, enabling significantly more parallelism and shorter training times.

Dependents

These beliefs depend on this one:

• IN lstm-transformer-parallelism-tradeoff — The LSTM-to-Transformer transition traded the ability to learn dependencies across unlimited timesteps for massively parallel training, a sequential-depth-for-parallel-breadth exchange.
• IN transformer-gpu-synergy-explains-dominance — Transformer dominance is partly explained by hardware synergy: eliminating sequential recurrence enables massive parallelism, which GPUs — the dominant ML training hardware — are specifically designed to exploit.