llama-4-moe-17b-active-128-experts
IN premise — summaries/2026/08/24/wiki-LLaMA-chunk-3.md
Created 2026-08-24T17:11:15+00:00
Llama 4 (April 2025) uses a Mixture-of-Experts architecture; the model identifier Llama-4-Maverick-17B-128E denotes 17B active parameters across 128 experts.
Summary
Llama 4 is built from 128 separate sub-networks called experts, but for any given input only a small subset is activated, so roughly 17 billion parameters do the actual work per token. This means the model has far more total capacity than it uses at once, trading full utilization for faster, cheaper inference while keeping a large overall knowledge footprint.