shared-language-pivot-affects-cross-lingual-concept-geometry
IN premise — summaries/2026/08/24/park-2023-linear-representation-s1-long-live-the.md
Created 2026-08-24T17:11:01+00:00
Under the causal inner product in LLaMA-2, the English→French concept has high inner product with French→German and French→Spanish (sharing 'French' as a pivot language) but remains nearly orthogonal to German→Spanish, while under the Euclidean inner product English→French has low inner product with other language concepts.
Summary
How the model organizes its knowledge of language pairs depends on which way you measure similarity: when you use a measure that respects causal direction, language pairs that share a common language (like English-to-French and French-to-German) cluster together, while pairs with no overlap (German-to-Spanish) sit far apart, but under a standard geometric distance the same English-to-French concept actually looks isolated from the others. This implies that the model's multilingual architecture is structured around shared pivot languages as natural computational hubs, and that conclusions about how "close" different language abilities are to each other can flip completely depending on the geometry you assume.