kandpal-2023-loglinear-r2-098-natural-questions
IN premise — summaries/2026/08/24/kandpal-2023-long-tail-knowledge-s3-lm-accuracy-depends-on-relevant.md
Created 2026-08-25T02:58:06+00:00
BLOOM accuracy on rare Natural Questions instances (<100 relevant documents) follows a log-linear trend with model parameter size with R² = 0.98.
Summary
When a question is genuinely hard to answer because fewer than 100 relevant documents exist, BLOOM's accuracy improves in a near-perfectly predictable way as the model gets bigger. This tight fit means you can confidently extrapolate performance to even larger models on rare, data-scarce questions, rather than guessing how well scaling will help in that difficult regime.