dominant-paradigms-empirically-fragile-and-transient
IN derived (depth 2)
Created 2026-06-21T10:09:45+00:00 · Reviewed 2026-06-21T15:37:01+00:00
The most successful ML paradigms are simultaneously dominant and fragile — pretrain-then-finetune is standard practice yet empirically hurtful in some transfer settings, GANs dominated generative modeling for years yet were displaced by diffusion — suggesting that current best practices are locally optimal recipes liable to succession rather than fundamental principles.
Justifications
SL — Two of ML's most successful paradigms each contain the seeds of their own displacement
Antecedents (all must be IN):
- IN pretraining-finetune-dominant-but-fragile — The pretrain-then-finetune paradigm is dominant for modern deep learning — underpinning both BERT and GPT — but is empirically fragile, as pretraining can actually hurt performance when strong data augmentation or self-training alternatives are available.
- IN generative-modeling-paradigm-succession — Generative modeling has seen a significant shift: GANs, introduced in 2014, were largely supplanted by diffusion models (such as DALL-E 2 and Stable Diffusion) from approximately 2022 onward, particularly for text-to-image generation, suggesting that even widely adopted adversarial training frameworks can be overtaken by alternative approaches.
Dependents
These beliefs depend on this one:
- IN ml-conceptual-foundations-doubly-unstable — ML's conceptual foundations are doubly unstable — the classical paradigm taxonomy (supervised/unsupervised/RL) is dissolving as modern pipelines combine all three, while even dominant paradigms like GANs and pretrain-finetune prove empirically fragile and transient — suggesting that ML's organizing categories are descriptive conveniences rather than natural kinds.
- IN ml-paradigm-impermanence-doubly-determined — No current ML paradigm can persist: theoretical completeness is demonstrated insufficient for survival (GANs' closed-form analysis didn't prevent displacement by diffusion models), and empirical dominance is independently fragile (pretrain-finetune is standard yet empirically hurtful in some settings) — paradigm impermanence is overdetermined by both theoretical and empirical evidence.
- IN modern-pretraining-dominant-but-transient — Modern pretraining as industrial-scale transfer learning is simultaneously the most successful ML methodology and the most likely to be displaced — its dominance rests on empirically fragile foundations (pretraining can hurt), and the broader pattern of paradigm succession (GANs→diffusion) suggests today's self-supervised pipelines are transient.
- OUT transfer-learning-resolves-paradigm-crisis — Transfer learning would resolve ML's paradigm dissolution crisis by providing a formal framework that bridges the classical supervised/unsupervised boundary, offering principled understanding of modern multi-paradigm pipelines rather than treating paradigm mixing as theoretically unprincipled.
- IN unsupervised-generative-evolution-validates-paradigm-succession — The evolution of unsupervised neural network architectures (Hopfield → Boltzmann → RBM → VAE) and the succession of generative model families (GANs → diffusion models) illustrate a recurring pattern in which dominant ML paradigms can be displaced by successors that may draw on predecessor ideas but achieve prominence through different approaches. This is consistent with the observation that even widely adopted paradigms appear to be locally optimal practices liable to succession rather than permanent foundations.