capacity-systematically-displaces-structure-across-all-domains
IN derived (depth 9)
Created 2026-06-21T14:03:57+00:00 · Reviewed 2026-06-21T15:37:01+00:00
Evidence from multiple ML domains — game-playing replacing explicit search with neural capacity, generative modeling's succession from structured to capacity-intensive architectures, and GPU-scalable neural networks displacing analytically elegant but non-parallelizable frameworks like SVMs — suggests a recurring pattern where hardware-driven capacity scaling favors architectures that trade structural understanding for raw capacity. This pattern appears across enough domains to indicate a broad tendency in ML evolution, though characterizing it as a universal law goes beyond what these specific cases establish.
Justifications
SL — Game-playing (search→capacity) combined with scalability-trumps-elegance (across all architecture selection) generalizes from specific instances to a universal displacement law.
Antecedents (all must be IN):
- IN search-replaceability-validates-capacity-over-structure — Game-playing's demonstration that explicit search can be complemented or replaced by sufficient model capacity suggests a pattern that may parallel the generative paradigm succession (Hopfield→Boltzmann→RBM→VAE→GAN→Diffusion) — in both domains, pragmatic selection appears to favor approaches that leverage increasing hardware capacity, though the game-playing evidence shows hybrid and pure neural paradigms remain viable depending on computational structure, and the generative succession is driven by a capability-adoption/displacement-before-resolution dynamic rather than a simple capacity-replaces-structure trajectory.
- IN scalability-trumps-elegance-in-ml — Hardware-architecture co-evolution favored architectures that could exploit parallelism (neural networks) over mathematically complete frameworks with limited parallelism benefits (SVMs). SVMs offered convex guarantees, kernel elegance, and sparse analytical solutions — a degree of mathematical closure few ML paradigms achieve — but neural networks' ability to scale with massive compute increases (300,000x from AlexNet to AlphaZero) was a significant factor in deep learning's dominance. This suggests engineering scalability became a major selection criterion for ML prominence, though the relative importance of compute scaling versus algorithmic innovation remains unestablished.
Dependents
These beliefs depend on this one:
- IN compute-scaling-quantifies-structure-displacement-rate — The 300,000x compute increase from AlexNet to AlphaZero (doubling every 3.4 months) provides a quantitative measure for the rate at which capacity growth has accompanied the displacement of structured mechanisms. The observed pattern — where increases in compute coincide with replacement of components like tree search, handcrafted features, and symbolic rules by neural capacity — suggests that structure displacement operates as an exponential process, though the precise relationship between each order of magnitude of compute and specific structural replacements is an observed correlation rather than a confirmed causal law.