uniform-superposition-regular-polytopes
IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition-chunk-6.md
Created 2026-08-25T02:58:00+00:00
In uniform superposition (equal importance and sparsity), optimal feature directions organize into vertices of regular polytopes (e.g., pentagon for 5 features in 2D, tetrahedron for 4 in 3D).
Summary
When every feature is given equal weight and kept sparse, they naturally spread out to form the most symmetric arrangement the space allows, like the evenly spaced corners of a regular shape. This means the system's internal geometry is predictable and maximally balanced, so how features interact follows clean, regular patterns rather than arbitrary ones.