sae-topk-architecture-used
IN premise — summaries/2026/08/24/templeton-2024-scaling-monosemanticity-chunk-1.md
Created 2026-08-25T02:58:36+00:00
Anthropic's scaling monosemanticity work on Claude 3 Sonnet uses TopK SAEs, which enforce sparsity more strictly than L1-regularized SAEs by retaining only the top-K activations in the sparse code.
Summary
Anthropic's interpretability work on Claude 3 Sonnet uses a method that hard-caps how many internal components can be active for any given input, rather than just softly discouraging extra activations. This means the features they find are defined by a strict "keep the top K, drop the rest" rule, which produces cleaner one-concept-per-feature results but also imposes a rigid structure that earlier, softer-sparsity methods did not.