float16-parameter-size-2bytes

IN premise — entries/2026/06/21/wiki-Large_language_model-chunk-1.md

Created 2026-06-21T09:50:09+00:00

One float16 parameter requires 2 bytes of storage, so 1 billion float16 parameters require approximately 2 GB.

Summary

This is the basic unit conversion that turns "model size in parameters" into "actual memory needed," since each float16 value is 2 bytes, a 1-billion-parameter model occupies roughly 2 GB. Every memory-budget, hardware-capacity, or deployment decision in the system ultimately rests on this arithmetic.

Dependents

These beliefs depend on this one:

IN llm-resource-consumption-quantifiable-across-three-dimensions — LLM resource consumption is precisely quantifiable across three independent dimensions — memory (2 bytes per float16 parameter), compute (6 FLOPs per parameter per token for training, 1-2 for inference), and energy (0.002-0.05 Wh per prompt depending on task type) — enabling principled cost modeling for deployment decisions.