training-identifies-representations-only-up-to-affine-transform

IN premise — summaries/2026/08/24/park-2023-linear-representation-s2-the-linear-representation-hypothesis.md

Created 2026-08-24T17:11:02+00:00

Training an LLM identifies the embedding and unembedding representations only up to an invertible affine transformation (γ(y) → Aγ(y) + β, λ(x) → A⁻ᵀλ(x)), which preserves all softmax probabilities, meaning no canonical Euclidean inner product is selected by training alone.

Summary

Training fixes the model's predictions but not the unique geometry of its internal word vectors, because many different arrangements of those vectors (rotated, stretched, shifted) produce identical outputs. This means the model never commits to a single "true" angle or distance between concepts, so any geometric reading of representation space — like saying two ideas are 40 degrees apart — rests on an arbitrary coordinate choice the training process did not select.