mistral-sae-1b-tokens-datasets

Summary

The interpretability layer built on top of Mistral 7B was trained on over a billion text examples pulled from two specific corpora, the Pile and Alpaca, so the features it extracts will reflect whatever kinds of content those datasets emphasize. This matters because it sets a boundary on what the system can interpret: concepts or patterns that are underrepresented in those two sources may be invisible to the autoencoder, and any downstream reasoning about model behavior inherits that blind spot.