maverick-codistilled-from-behemoth

IN premise — entries/2026/06/21/wiki-LLaMA-chunk-1.md

Created 2026-06-21T09:50:09+00:00

Llama 4 Maverick was codistilled from the unreleased Behemoth model (~2T total parameters), while Scout was trained from scratch

Summary

Maverick didn't learn the world on its own; it was compressed from a much larger, never-released teacher model called Behemoth, so its capabilities and quirks are inherited shortcuts rather than independent discoveries. Scout, by contrast, was trained from the ground up, meaning the two models in the same Llama 4 family actually have different lineages, which affects how you'd expect their strengths, failure modes, and behavior to diverge.

Dependents

These beliefs depend on this one:

IN distillation-validated-across-full-scale-spectrum — Knowledge distillation is validated as a scale-invariant capability across the full spectrum of language model sizes: from BERT-scale (DistilBERT retaining 95% performance at 60% of parameters) to frontier-scale (Llama 4 Maverick codistilled from the unreleased ~2T-parameter Behemoth), demonstrating that larger models reliably compress their capability into smaller ones regardless of absolute scale.