park-2025-centering-alone-insufficient
IN premise — summaries/2026/08/24/park-2024-categorical-hierarchical-concepts-sR-references-chunk-1.md
Created 2026-08-25T02:58:25+00:00
Centering unembedding vectors alone (preserving the Euclidean inner product) does not yield hierarchical orthogonality; both centering and whitening (covariance inversion) are required for the Causal Inner Product.
Summary
Simply shifting vectors to their mean is not enough to make the output-layer geometry reflect true hierarchical independence between concepts; you also have to undo the correlated stretch between dimensions. Without both steps, any similarity or orthogonality you measure will silently bake in spurious correlations, leading to wrong causal interpretations of how concepts relate in the model.