llama-uses-swiglu-rope-rmsnorm

OUT premise — entries/2026/06/21/wiki-LLaMA.md

Created 2026-06-21T09:50:09+00:00

Llama uses SwiGLU activation (not GeLU), rotary positional embeddings/RoPE (not absolute), and RMSNorm (not LayerNorm), differentiating it from GPT-3's architecture.