park-2025-softmax-invariance-whitening

IN premise — summaries/2026/08/24/park-2024-categorical-hierarchical-concepts-s2-using-this-result-we-show-that-semantic-hierarchy-between-co.md

Created 2026-08-25T02:58:25+00:00

The whitening/space-unification transformation A(γ(y) − γ̄₀) with corresponding λ̃(x) = A^(−⊤) λ(x) does not change softmax output probabilities P(y|x).

Summary

You can rescale and rotate the internal representation space (the "whitening" step) and, as long as you mirror that change on the parameter side with the exact inverse transpose, the final probability scores the model produces for each class stay completely identical. This matters because it means the system is free to normalize or reorient its internal geometry for numerical stability or interpretability without ever having to retrain or worry about altering predictions.