anthropic-sae-512-neuron-mlp-architecture

Summary

This establishes the experimental setup for the Bricken et al. interpretability work: a deliberately small, single-layer transformer with a modest 512-feature feed-forward block, trained on a large but not enormous corpus. Any downstream findings about how the model processes information are bounded by this simple architecture, so conclusions drawn from it may not transfer directly to deeper or larger production models.