llama-context-window-progression

IN premise — entries/2026/06/21/wiki-LLaMA-chunk-1.md

Created 2026-06-21T09:50:09+00:00

Llama context windows grew from 2K (Llama 1) to 4K (Llama 2) to 128K (Llama 3.1) to 10M (Llama 4 Scout)

Summary

Each generation of Llama has dramatically expanded how much text the model can process in a single pass, growing roughly 5,000-fold from the first release to the latest. This matters because it means newer models can now swallow entire books, full codebases, or long multi-turn conversations without losing track of earlier content, fundamentally changing what kinds of tasks they can handle without external memory or chunking.