training-pipeline-maturity-masks-scaling-asymmetry
IN derived (depth 3)
Created 2026-06-21T10:12:39+00:00 · Reviewed 2026-06-21T14:41:08+00:00
The LLM training pipeline's maturation into a standardized engineering discipline (SFT → reward model → PPO, with known costs and reproducible stages) masks a fundamental asymmetry: pretraining benefits most from scaling data volume while alignment benefits most from scaling model size — meaning the same pipeline optimizes in opposite directions at different stages, and uniform scaling strategies are suboptimal.
Summary
Because the training pipeline now looks like a repeatable engineering process, teams tend to scale it uniformly, but that hides a real problem: pretraining wants more data while alignment wants a bigger model. This means the same pipeline is pulling in opposite directions at different stages, and a single scaling strategy will underperform in at least one of them.
Justifications
SL — Pipeline standardization (depth-2) conceals inverted scaling priorities (depth-2) between its own stages
Antecedents (all must be IN):
- IN training-pipeline-is-mature-engineering-discipline — The LLM training pipeline has matured into a fully specified engineering discipline with standardized stages (pretraining → SFT → alignment), complete mathematical specification of the alignment phase (Bradley-Terry loss, KL penalty, PPO clipping), and calibrated cost models (6 FLOPs/param/token) enabling predictable budgeting from BERT's $500 to PaLM's millions.
- IN pretraining-and-alignment-have-inverted-scaling-priorities — Pretraining and alignment exhibit inverted scaling priorities: in pretraining, data volume outweighs parameter count (Chinchilla, Llama 1 13B beating GPT-3 175B), while in RLHF reward modeling, scaling model size is more effective than scaling comparison dataset size — suggesting different phases of the training pipeline are bottlenecked by fundamentally different resources.
Dependents
These beliefs depend on this one:
- IN engineering-maturity-systematically-outpaces-theoretical-understanding — The LLM field is systematically characterized by engineering maturity outrunning theoretical understanding — standardized pipelines, reproducible results, and practical compensations consistently succeed at every level of the stack despite fundamental theoretical insufficiency that would normally preclude confidence.
- IN parameter-redundancy-enables-reliability-despite-theory-gaps — Parameter redundancy may help explain why the mature training pipeline functions reliably despite unresolved scaling asymmetries — over-parameterized models can absorb suboptimal choices across pipeline stages that optimize in opposite directions (data-scaling for pretraining vs. model-scaling for alignment), providing architectural slack that the compression literature suggests accounts for a substantial fraction of parameters.
- IN training-pipeline-masks-fundamental-capacity-inversion — The mature training pipeline's standardized stages mask a fundamental capacity inversion: pretraining benefits from parameter redundancy (over-parameterized models remain compressible), while alignment is bottlenecked by reward model capacity (scaling the reward model matters more than scaling data), and this asymmetry is hidden by the pipeline's apparent end-to-end reproducibility.