saey-target-architecture-one-layer-512-mlp

Summary

The Bricken et al. 2023 monosemanticity work was demonstrated on a deliberately tiny, single-layer model with only 512 hidden neurons, not on a full-scale language model. This matters because it means the paper's results prove the SAE/monosemanticity approach works in principle on a simple toy architecture, but do not by themselves establish how the method scales to the large, multi-layer models actually used in practice.