euclidean-inner-product-partially-works-due-to-isotropic-covariance

IN premise — summaries/2026/08/24/park-2023-linear-representation-s1-long-live-the.md

Created 2026-08-24T17:11:01+00:00

The standard Euclidean inner product partially captures concept separability in LLaMA-2's unembedding space because the model's unembedding weights exhibit approximately isotropic covariance (likely from weight initialization or implicit regularization), making separable concepts roughly orthogonal, but it produces false positives (e.g., frequent→infrequent correlating with many separable concepts) that the causal inner product corrects.

Summary

The simple dot-product trick for measuring how distinct a concept is in LLaMA-2's output layer gives roughly the right answer for many pairs of words because the model's output weights happen to be spread out evenly in all directions, making unrelated concepts land at right angles. But that same flat geometry also makes very common words look separable from a bunch of rare ones when they really aren't, so any analysis relying on the plain dot product will overstate how cleanly the model separates concepts unless you use the more careful causal version.