park-2023-riesz-map-unifies-embedding-unembedding
IN premise — summaries/2026/08/24/park-2023-linear-representation-s3-inner-product-for-language-model.md
Created 2026-08-24T17:11:03+00:00
Under the causal inner product, the Riesz map γ̄ ↦ ⟨γ̄, ·⟩_C sends each unembedding representation γ̄_W exactly to its embedding representation λ̄_W; in the transformed space with A = M^{1/2}, embeddings and unembeddings are literally equal (g̃_W = l̃_W) and the Euclidean inner product becomes the causal one.
Summary
What were traditionally tracked as two separate objects — a layer's embedding and its unembedding — are actually the same quantity once you measure them with the right geometric ruler. This means the system can treat them as a single unified entity rather than maintaining parallel copies, simplifying analysis and removing a whole class of inconsistency bugs that arise when the two drift apart.
Dependents
These beliefs depend on this one:
- OUT riesz-map-as-canonical-semantic-isomorphism — The Riesz map under the causal (whitened) inner product is the unique canonical isomorphism that makes embedding and unembedding semantically identical, thereby elevating "meaning" from a model-specific activation pattern to a well-defined algebraic object in a model-independent vector space