superposition-requires-feature-sparsity

IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition.md

Created 2026-08-25T02:58:01+00:00

With dense features, a model collapses to an orthogonal basis of top-k features (PCA-like); with sparse features, the model exploits rare co-occurrence to pack more than d features into d dimensions (superposition).

Summary

This is the foundational observation that superposition only works because concepts in the data are mostly absent from any given input. If every concept showed up in every input, the model would be forced to assign each one its own separate direction, giving no more than as many concepts as dimensions, and the whole mechanism of packing many ideas into a small space would be impossible.