paradigm-survival-determined-by-scalability-not-theory

IN derived (depth 4)

Created 2026-06-21T10:13:05+00:00 · Reviewed 2026-06-21T15:37:01+00:00

Mathematical completeness and theoretical elegance are neither necessary nor sufficient for paradigm survival in ML — GANs had the most complete analytical characterization yet were eclipsed by diffusion models, SVMs had convex guarantees yet were outscaled by neural networks, while theoretically less grounded approaches that scaled with hardware thrived.

Justifications

SL — depth-4 — two independent depth-3 conclusions (one from classical ML, one from generative modeling) converge on scalability as the sole selection pressure

Antecedents (all must be IN):

IN scalability-trumps-elegance-in-ml — Hardware-architecture co-evolution favored architectures that could exploit parallelism (neural networks) over mathematically complete frameworks with limited parallelism benefits (SVMs). SVMs offered convex guarantees, kernel elegance, and sparse analytical solutions — a degree of mathematical closure few ML paradigms achieve — but neural networks' ability to scale with massive compute increases (300,000x from AlexNet to AlphaZero) was a significant factor in deep learning's dominance. This suggests engineering scalability became a major selection criterion for ML prominence, though the relative importance of compute scaling versus algorithmic innovation remains unestablished.
IN theoretical-completeness-no-guarantee-of-paradigm-durability — Theoretical completeness does not guarantee paradigm durability — GANs had a notably complete analytical characterization (closed-form optimal discriminator, JSD minimization proof, unique equilibrium) yet were largely supplanted by diffusion models from approximately 2022 onward, suggesting that factors beyond theoretical elegance — possibly including training reliability — may play a significant role in determining which paradigms persist.

Dependents

These beliefs depend on this one:

IN architecture-selection-two-stage-filter — ML architecture selection operates as a two-stage filter: data geometry determines which inductive biases are effective (first filter), and hardware scalability determines which effective architectures survive (second filter) — scalability can veto geometric fit but not vice versa, explaining why theoretically superior architectures are routinely displaced.
IN hardware-economics-primary-selection-pressure — Hardware economics is a primary selection pressure in ML's evolution — co-evolution with theory shapes choices from macro architecture decisions to micro implementation tradeoffs, and paradigm survival correlates more strongly with scalability than with theoretical completeness, suggesting that compute availability significantly shapes what ML becomes, alongside but often outweighing mathematical insight.
IN math-determines-failure-mode-economics-determines-timing — Mathematical impossibility results and economic selection pressures play orthogonal roles in paradigm evolution — GAN Nash impossibility (Farnia & Ozdaglar 2020) mathematically necessitated the specific failure mode (training instability, mode collapse) but did not determine adoption or displacement timing, which was governed by scalability economics; mathematics determines HOW paradigms fail while economics determines WHEN.
IN nlp-validates-scalability-over-theory-selection — NLP's paradigm trajectory provides partial independent support for hardware scalability as a primary factor in paradigm survival — the symbolic-to-statistical-to-neural succession and the RNN-to-LSTM-to-Transformer architectural evolution both correlate with hardware capability, though hardware-driven selection was one of several primary factors (alongside attention mechanisms and memory-parallelism tradeoffs) rather than the sole determinant, offering a domain-specific case consistent with the general pattern that scalability outweighs theoretical elegance.
IN pragmatism-recursive-across-discovery-displacement-and-selection — Pragmatism operates recursively at three nested levels in ML — it governs which mechanisms are discovered (enabling cross-field convergence on mathematical necessities), which implementations of those mechanisms survive (GRU's simplification of LSTM), and which paradigms hosting those implementations persist (scalability over elegance determines evolutionary success) — meaning pragmatism is not merely a selection pressure on the field but a fractal organizational principle replicated at every level of abstraction.
IN quadratic-attention-drives-architectural-succession-pressure — The Transformer's quadratic attention cost creates permanent architectural succession pressure — just as RNNs were displaced by Transformers for failing to parallelize, Transformers face displacement pressure from linear-complexity alternatives (Mamba, RWKV, Reformer), confirming that hardware scalability determines paradigm survival applies reflexively to the currently dominant architecture.
IN rnn-turing-completeness-purest-case-of-theoretical-irrelevance — RNNs' proven Turing-completeness (Siegelmann & Sontag 1994) — the strongest computational-theoretic result for any neural architecture family — is the purest demonstration that theoretical computational power is irrelevant to paradigm survival, as Transformers displaced RNNs solely through superior hardware utilization despite possessing strictly less formal computational power.
OUT transformer-dominance-indefinitely-sustainable — Transformer architectural dominance would be indefinitely sustainable — paradigm survival is determined by hardware scalability not theoretical elegance, and transformers' unique combination of architectural flexibility (encoder-only/decoder-only/encoder-decoder specialization) with GPU parallelism synergy creates a deepening competitive moat that no alternative can breach on the current hardware landscape.