engels-2024-theorem1-lower-bound
IN premise — summaries/2026/08/24/engels-2024-not-all-features-linear-sR-references.md
Created 2026-08-25T02:58:03+00:00
In a d-dimensional space, the number of pairwise δ-orthogonal d′-dimensional subspaces is bounded below by Ω(e^{C(d/d′²)δ²}/d) (Engels et al., ICLR 2025, Appendix A).
Summary
In high-dimensional space, you can fit an exponentially large number of lower-dimensional subspaces that are all nearly uncorrelated with one another, and the count grows sharply as the ratio of total dimensions to sub-space dimensions increases. This matters because it guarantees there is more than enough geometric room to pack in many nearly-independent directions, which underpins any technique that relies on separating components, from neural embeddings to error-correcting codes.