grokking-memorize-then-generalize

IN premise — entries/2026/06/21/wiki-Large_language_model-chunk-2.md

Created 2026-06-21T09:50:09+00:00

Grokking is the phenomenon where a model first memorizes training data (overfitting), then suddenly learns the underlying algorithm and generalizes, discovered via mechanistic interpretability of modular arithmetic models.

Summary

A model can first just memorize its training examples, then suddenly extract the underlying rule and start generalizing to problems it has never seen. This matters because overfitting is not always a dead end; it can be a necessary stepping stone, and stopping training at the point of best memorization would miss the real understanding that comes next.

Dependents

These beliefs depend on this one:

IN grokking-implies-training-traverses-security-vulnerable-memorization-phase — Grokking's memorize-then-generalize dynamic — where models first memorize training data before suddenly discovering generalizable patterns — combined with memorization's dual-use nature (knowledge source and extraction attack surface) suggests that model training may traverse a memorization-heavy phase where training data is more extractable before generalization emerges. However, the evidence for this comes from mechanistic interpretability of modular arithmetic models and memorization measurements at the GPT-2 scale, so whether this phase constitutes a universal security-vulnerable window across all training regimes, and whether earlier checkpoints systematically embed more extractable data than later ones, remains an extrapolation beyond what the current antecedents directly establish.
OUT grokking-memorization-phase-manageable-under-controlled-training — Grokking's memorize-then-generalize dynamic implies that the security-vulnerable memorization phase is a transient training state that resolves under continued training — models move from memorization (maximal data extractability) to generalization (compressed, abstract representations), making the vulnerability window manageable under controlled training conditions where intermediate checkpoints are secured.
OUT llm-learning-involves-genuine-phase-transitions — LLM capability acquisition involves genuine discontinuous phase transitions — both grokking (sudden generalization after memorization within training) and emergent abilities (capabilities appearing at scale thresholds) — rather than smooth, predictable improvement curves.