decoder-only-dominance-is-permanent-architectural-consensus
OUT derived (depth 5)
Created 2026-06-21T13:28:06+00:00
Decoder-only architecture's dominance — driven by data scaling advantage and confirmed by the innovation frontier's shift from macro-architecture to micro-architectural configuration — represents a permanent architectural consensus, with no remaining structural incentive to revisit the encoder/decoder choice.
Justifications
SL — Decoder-only dominance appears permanent unless non-transformer alternatives like RWKV achieve frontier quality
Antecedents (all must be IN):
- IN decoder-only-dominance-driven-by-data-scaling-advantage — Decoder-only architectures became dominant in the scaling era, coinciding with evidence that data volume matters more than parameter count for model performance. Autoregressive next-token prediction's ability to leverage massive unsupervised text corpora may have been a contributing factor in this dominance.
- IN innovation-frontier-shifted-from-architecture-to-configuration — The LLM innovation frontier has shifted from macro-architecture selection (settled: decoder-only transformer via data-driven dominance) to micro-architecture configuration (unsettled: tokenization, positional encoding, normalization) and data strategy optimization.
Unless (any of these IN defeats this justification):
- IN rwkv-rnn-with-transformer-training — RWKV (Peng et al., 2023) reinvents RNNs with Transformer-era training techniques, avoiding the quadratic attention complexity while attempting comparable performance.