superposition-requ-rel-relu-necessary
IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition-chunk-2.md
Created 2026-08-25T02:57:58+00:00
The Elhage et al. 2022 paper demonstrates that ReLU (or a similar nonlinear filter) is necessary to recover individual superposed features; without it, the model cannot separate overlapping feature signals in shared dimensions.
Summary
When a neural network packs many distinct concepts into the same small set of numbers, a nonlinear gate like ReLU is what lets you pull those concepts back out cleanly; without it, the overlapping signals blur together and become irrecoverable. This means the nonlinearity is not a minor training detail but a structural requirement — any model that relies on sharing dimensions across many features depends on that gate to keep the features individually readable.