uniform-superposition-converges-digons
IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition-chunk-7.md
Created 2026-08-25T02:58:00+00:00
In the uniform-superposition setup (equal importance, equal sparsity), features converge into a small number of regular polytopes (e.g., digons) with fractional dimensionalities.
Summary
When every feature is given the same weight and the same level of sparsity, the features don't stay scattered and chaotic. Instead they self-organize into just a handful of simple geometric patterns, and those patterns turn out to have non-integer (fractional) dimensions, meaning the system has a small, predictable geometric skeleton you can reason about rather than an endlessly complicated shape.