llama-exemplifies-craft-discipline-architecture-by-assembly
IN derived (depth 9)
Created 2026-06-21T12:56:38+00:00 · Reviewed 2026-06-21T14:41:08+00:00
Llama's architecture-by-assembly strategy — adopting proven refinements (SwiGLU, RoPE, RMSNorm) from independent prior research while focusing effort on data scaling — exemplifies the craft discipline's empirical epistemology: rather than innovating architecturally, Meta assembled the best empirically validated components and invested in the scaling lever the field had empirically identified as dominant.
Justifications
SL — Llama's design philosophy is the craft discipline's epistemology in action — empirical assembly over theoretical innovation
Antecedents (all must be IN):
- IN llama-optimized-architecture-for-data-scaling-strategy — Llama's architecture adopts proven refinements from prior independent research (SwiGLU from Shazeer 2020, RoPE from Su et al. 2021, RMSNorm from Zhang & Sennrich 2019) rather than introducing novel components, while its training data volume scaled dramatically from 1.4T to 40T tokens across four generations — a pattern consistent with concentrating effort on data scaling over architectural novelty.
- IN llm-field-is-fundamentally-craft-discipline — The LLM field is fundamentally a craft discipline: both its most valuable structural properties (cross-boundary innovation, parameter redundancy) and its deepest barriers (tacit deployment knowledge, experiential prerequisites) are discovered and transmitted empirically, not through formal theory — meaning neither mastery nor failure modes are accessible through documentation alone.
Dependents
These beliefs depend on this one:
- OUT craft-architectural-convergence-guarantees-behavioral-stability — The craft discipline's empirical methodology — producing convergent architectural choices across independent research groups (SwiGLU, RoPE, RMSNorm adopted by Llama, PaLM, and others via craft-by-assembly) combined with innovation transcending organizational boundaries — guarantees behaviorally stable architectures through empirical selection pressure.