chinchilla-scaling-law-constants
IN premise — entries/2026/06/21/wiki-Large_language_model-chunk-2.md
Created 2026-06-21T09:50:09+00:00
The Chinchilla scaling law is L = A/N^alpha + B/D^beta + L0 with alpha=0.34, beta=0.28, L0=1.69, and training cost C = 6·N·D FLOPs.
Summary
These are the specific numbers that let the system calculate, for any chosen model size and training budget, exactly how much quality to expect and how much compute it will cost. Without them fixed, any trade-off reasoning between building a bigger model, collecting more data, or spending more compute would have no quantitative anchor.
Dependents
These beliefs depend on this one:
- OUT chinchilla-balance-prescribes-optimal-resource-allocation — Chinchilla's prescription to scale parameters and data in equal proportion provides the optimal training resource allocation strategy.
- IN chinchilla-grounds-scaling-in-information-theory — Chinchilla research established that language model quality directly measures data compression capability (compressing ImageNet to 43% vs PNG's 58%), grounding compute-optimal scaling laws in information-theoretic foundations rather than purely empirical curve-fitting.