training-pipeline-masks-fundamental-capacity-inversion

IN derived (depth 4)

Created 2026-06-21T11:17:49+00:00 · Reviewed 2026-06-21T14:41:08+00:00

The mature training pipeline's standardized stages mask a fundamental capacity inversion: pretraining benefits from parameter redundancy (over-parameterized models remain compressible), while alignment is bottlenecked by reward model capacity (scaling the reward model matters more than scaling data), and this asymmetry is hidden by the pipeline's apparent end-to-end reproducibility.

Summary

The standard LLM training pipeline looks like a uniform engineering process you can scale evenly at every stage, but it actually hides a contradiction: the early phase needs more data and tolerates excess parameters, while the alignment phase is limited by how big its reward model is, not by data volume. Treating the pipeline as a single scalable unit and applying the same growth strategy everywhere wastes compute where it doesn't matter and starves the stage that genuinely needs more capacity.

Justifications

SL — Pipeline reproducibility obscures that pretraining and alignment have opposite scaling bottlenecks

Antecedents (all must be IN):

IN training-pipeline-maturity-masks-scaling-asymmetry — The LLM training pipeline's maturation into a standardized engineering discipline (SFT → reward model → PPO, with known costs and reproducible stages) masks a fundamental asymmetry: pretraining benefits most from scaling data volume while alignment benefits most from scaling model size — meaning the same pipeline optimizes in opposite directions at different stages, and uniform scaling strategies are suboptimal.
IN capacity-bottleneck-inverts-between-pretraining-and-alignment — The transformer's known over-parameterization at the pretraining level (compression to 60% of parameters with 95% performance retained) combined with alignment's inverse priority (scaling reward model size matters more than data volume) reveals a systematic inversion: data volume is the bottleneck for pretraining while model capacity is the bottleneck for alignment.

Dependents

These beliefs depend on this one:

IN alignment-bootstrap-creates-circular-safety-assurance — The alignment system's circular dependency is deeper than its bootstrap origin: alignment is a product of the craft methodology it compensates for, AND the training pipeline masks the fundamental capacity inversion between pretraining and alignment — meaning the craft methodology that cannot formally verify safety also masks the very capacity asymmetry that makes alignment insufficient, creating a circular safety assurance where the evaluation method is blind to the failure mode it should detect.
IN craft-validation-masks-capacity-inversion — The craft discipline's empirical validation methodology actively masks the fundamental capacity inversion between pretraining and alignment: because the field validates by deployment outcomes rather than formal analysis, the training pipeline's standardized stages appear uniformly mature even though pretraining benefits from parameter redundancy while alignment faces strict capacity constraints requiring entirely different scaling strategies.