disentanglement-vs-superposition-research-distinction

IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition-chunk-12.md

Created 2026-08-25T02:57:59+00:00

Disentanglement research aims to impose a privileged basis on a latent space, whereas superposition research starts from an existing (neuron) basis and attempts to recover features that are superimposed within that basis.

Summary

These two research programs look similar on the surface but start from opposite directions: disentanglement tries to carve a latent space into clean, pre-chosen feature axes, while superposition work starts with the neurons as they are and tries to unpack the features that were crammed into them. Recognizing this distinction matters because it means the two approaches are not interchangeable shortcuts to the same goal, and conflating them can lead to drawing the wrong conclusions about what a model actually computes.