sae-algorithmic-improvement-bricken
IN premise — summaries/2026/08/24/templeton-2024-scaling-monosemanticity-chunk-10.md
Created 2026-08-25T02:58:36+00:00
A key algorithmic improvement in the scaling monosemanticity paper is multiplying the sparsity penalty by the decoder norm and removing the unit-norm constraint on decoder vectors, proposed by Bricken and verified by Conerly and Templeton.
Summary
A specific training tweak for sparse autoencoders — scaling the sparsity penalty by decoder size and letting decoders vary in length rather than being locked to a fixed size — is attributed as Bricken's proposal, independently confirmed by Conerly and Templeton. This pins down the provenance of a key algorithmic step, so any downstream reasoning about the method's origins or reliability can trace exactly who contributed what.