llm-learning-involves-genuine-phase-transitions

OUT derived (depth 1)

Created 2026-06-21T09:57:37+00:00

LLM capability acquisition involves genuine discontinuous phase transitions — both grokking (sudden generalization after memorization within training) and emergent abilities (capabilities appearing at scale thresholds) — rather than smooth, predictable improvement curves.

Justifications

SL — holds unless emergent abilities are shown to be measurement artifacts of metric choice

Antecedents (all must be IN):

IN grokking-memorize-then-generalize — Grokking is the phenomenon where a model first memorizes training data (overfitting), then suddenly learns the underlying algorithm and generalizes, discovered via mechanistic interpretability of modular arithmetic models.
IN emergent-abilities-discontinuous-scale — Emergent abilities in LLMs appear discontinuously at certain scale thresholds, not linearly (Wei et al., 2022, arXiv:2206.07682)

Unless (any of these IN defeats this justification):

IN emergent-abilities-metric-artifact-debate — The appearance of emergent abilities in LLMs depends on metric choice: accuracy metrics show step-function discontinuities while log-probability metrics show smooth scaling curves (Schaeffer et al.).