park-2023-d-free-degrees-of-freedom
IN premise — summaries/2026/08/24/park-2023-linear-representation-s3-inner-product-for-language-model.md
Created 2026-08-24T17:11:03+00:00
Causal orthogonality imposes d(d−1)/2 constraints on the d×d metric matrix M, but M has d(d+1)/2 free parameters, leaving exactly d free degrees of freedom; the paper selects the diagonal D = I_d for concreteness but acknowledges non-uniqueness.
Summary
The math shows that after enforcing causal orthogonality on a d-dimensional metric, exactly d parameters remain unconstrained — the authors pin them down by choosing an identity matrix, but they openly note this is one valid option among many. This matters because any downstream results that depend on that specific diagonal choice could, in principle, be re-expressed under a different valid choice, so the identity-matrix fix is a convenience, not a unique requirement of the model.