sft-one-epoch-confirms-inverted-scaling-dynamics

IN derived (depth 3)

Created 2026-06-21T13:28:05+00:00 · Reviewed 2026-06-21T14:41:08+00:00

The InstructGPT paper's deliberate restriction of SFT training to a single epoch — explicitly to avoid overfitting — independently confirms the inverted scaling dynamics between pretraining and alignment: pretraining thrives on data volume (15T+ tokens in Llama 3) while alignment degrades from the same abundance, requiring intentional information restriction at the pipeline transition point.

Justifications

SL — SFT's 1-epoch restriction is direct evidence of the pretraining/alignment scaling inversion

Antecedents (all must be IN):

IN rlhf-sft-one-epoch-instructgpt — The InstructGPT paper recommends training the SFT stage for only 1 epoch to avoid overfitting.
IN pretraining-and-alignment-have-inverted-scaling-priorities — Pretraining and alignment exhibit inverted scaling priorities: in pretraining, data volume outweighs parameter count (Chinchilla, Llama 1 13B beating GPT-3 175B), while in RLHF reward modeling, scaling model size is more effective than scaling comparison dataset size — suggesting different phases of the training pipeline are bottlenecked by fundamentally different resources.