park-2025-direct-sum-space-decomposition
IN premise — summaries/2026/08/24/park-2024-categorical-hierarchical-concepts-s2-using-this-result-we-show-that-semantic-hierarchy-between-co.md
Created 2026-08-25T02:58:25+00:00
The combination of polytope representations and hierarchical orthogonality (Theorem 8) implies the full representation space decomposes as a direct sum of orthogonal subspaces, one per level of the hierarchy.
Summary
The full space of all representations breaks apart into clean, non-overlapping slices, one for each level in the hierarchy, so that what happens in one level never bleeds into another. This means the system can reason about each level independently, without having to track or compensate for cross-level interactions, which drastically simplifies both computation and verification.
Dependents
These beliefs depend on this one:
- OUT sae-neighborhood-as-directsum-navigation — The SAE feature neighborhood (Golden Gate Bridge → Alcatraz → San Francisco → California) is not merely a navigation path within a single categorical polytope but the operational traversal algorithm for the full direct-sum decomposition of the semantic space: it simultaneously navigates the categorical subspace (discrete concepts) and the hierarchical orthogonal subspace (WordNet parent-child structure), validated across Gemma-2B and LLaMA-3-8B.
- OUT space-decomposition-under-superposition — The full LLM semantic space admits a clean algebraic decomposition into categorical polytope subspaces (discrete concepts) and hierarchical orthogonality subspaces (graded taxonomic structure) within the covariance-geometric framework, providing a complete account of how discrete and graded meaning coexist in a single over-complete vector space
- OUT superposition-as-compositional-basis — Superposition is the fundamental compositional mechanism in LLMs: the 10–200× over-complete expansion (SAE), the key-value memory structure (ROME's W_fc/W_proj), and the direct-sum space decomposition (Park's polytope+orthogonality) are three independent geometric consequences of the same over-completeness.