rnn-theoretical-power-practically-accessible

OUT derived (depth 2)

Created 2026-06-21T10:06:02+00:00

RNNs' theoretical computational power (Turing-completeness, context-sensitive language recognition) is practically accessible for sequence modeling tasks, given LSTM's ability to learn over thousands of timesteps.

Justifications

SL — RNN theoretical power was practically relevant until Transformers provided a superior practical alternative

Antecedents (all must be IN):

IN rnn-theoretical-power-exceeded-practical-utility — RNNs possess strong theoretical computational power — Turing-completeness with rational weights (Siegelmann & Sontag 1994) and context-sensitive language recognition via LSTM that surpasses what HMM-based models achieve — while their sequential hidden-state processing, which updates state at each time step, represents an inherent constraint on parallelism.
IN lstm-learns-over-thousands-millions-timesteps — LSTM can learn dependencies across thousands or millions of time steps by allowing error to flow backward through unlimited virtual layers.

Unless (any of these IN defeats this justification):

IN transformer-replaced-recurrence-with-self-attention-2017 — The Transformer (Vaswani et al. 2017, 'Attention is All You Need') replaced recurrence with self-attention and became dominant for sequence modeling.