saey-256x-expansion-131072-features
IN premise — summaries/2026/08/24/bricken-2023-monosemanticity-chunk-1.md
Created 2026-08-25T02:57:54+00:00
A sparse autoencoder with a 256× expansion factor applied to a 512-neuron layer yields 131,072 learned features.
Summary
The network's architecture has been set so that a 512-neuron layer is expanded 256 times, locking in a total of 131,072 learned features. That fixed count is now a baseline the rest of the system depends on, shaping everything from computational cost to the number of interpretable signals available for reasoning.