saey-a1-autoencoder-4096-features-dead-ultralow
IN premise — summaries/2026/08/24/bricken-2023-monosemanticity-chunk-4.md
Created 2026-08-25T02:57:55+00:00
The A/1 autoencoder produces 4,096 features, of which 168 are dead (never active on 100M examples) and 292 are ultralow-density (active on fewer than 1 in 1M examples), leaving approximately 3,636 for analysis.
Summary
Out of the 4,096 output units in the A/1 autoencoder, roughly 460 are effectively wasted capacity: some never fire across 100 million examples and others are so rare they barely count as alive. That means any meaningful analysis of what the autoencoder actually learned should focus on the roughly 3,600 units that consistently participate, and the dead or near-dead units are a sign of design inefficiency rather than additional signal.