nanda-replication-codebase

IN premisesummaries/2026/08/24/bricken-2023-monosemanticity-chunk-8.md

Created 2026-08-25T02:57:56+00:00

Neel Nanda's open-source SAE replication codebase is hosted at https://github.com/neelnanda-io/1L-Sparse-Autoencoder and uses a 1-layer GELU language model

Summary

This pins down a specific, publicly available starting point for Sparse Autoencoder work: a GitHub repo from Neel Nanda that reproduces SAE features on a deliberately simple one-layer GELU model, so anyone in the system can point to concrete code rather than just a paper. The fact that it uses a minimal base model means the learned features are easier to interpret and the setup is lightweight enough to rerun or modify without heavy compute.