solving-superposition-defined-as-feature-enumeration
IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition-chunk-11.md
Created 2026-08-25T02:57:59+00:00
'Solving superposition' is defined in the Elhage et al. 2022 paper as achieving the ability to enumerate all features in a model, equivalently 'unfolding' superposition activations into a larger non-superposition space via compressed sensing.
Summary
In the Elhage et al. 2022 work, "solving superposition" is set to a very high bar: it means recovering every single feature packed into a model's neurons, not just a few, by mathematically unpacking the overlapping activations into a wider space where each feature stands on its own. This matters because it turns a vague goal of "understanding the model" into a concrete, testable target rooted in compressed-sensing theory, and it sets the benchmark against which any interpretability method would have to be judged in that framework.