llama-decoder-only-autoregressive

IN premise — entries/2026/06/21/wiki-LLaMA-chunk-1.md

Created 2026-06-21T09:50:09+00:00

Llama is a decoder-only autoregressive transformer architecture (like GPT-3), not encoder-decoder or encoder-only

Summary

Llama builds text the same way GPT does, predicting one piece at a time in a single left-to-right pass rather than first digesting a full input and then producing a separate output. This single-stream design means the model never gets to see what comes ahead, and every step of reasoning or generation is just "what's next," which constrains how it can restructure or revisit earlier content.

Dependents

These beliefs depend on this one:

IN decoder-only-won-the-scaling-race — Decoder-only architecture became the dominant paradigm for large language models, with GPT, Llama, and most frontier models choosing autoregressive causal modeling over encoder-only or encoder-decoder variants.