llama1-13b-outperformed-gpt3-175b

IN premise — entries/2026/06/21/wiki-LLaMA-chunk-1.md

Created 2026-06-21T09:50:09+00:00

Llama 1 13B outperformed GPT-3 175B on most NLP benchmarks, demonstrating the value of data scaling over parameter scaling

Summary

A model with roughly one thirteenth the parameters of GPT-3 beat it on most language tasks, which means that feeding a model more and better training data can matter more than simply making the model bigger. This challenges the assumption that bigger models are automatically better and suggests that resource budgets should weigh data quality and volume as heavily as raw parameter count.

Dependents

These beliefs depend on this one:

IN data-scaling-outweighs-parameter-scaling — Empirical results consistently show data volume matters more than parameter count: Chinchilla demonstrated models were undertrained, Llama 1 13B beat GPT-3 175B, and Llama 3 8B continued improving at 75x Chinchilla-optimal data.
OUT llama-is-practical-open-weight-alternative — Llama serves as a practical open-weight alternative to proprietary models, with competitive performance (13B beating GPT-3 175B) and local deployment infrastructure (llama.cpp).