thing-to-part-failed-linear-representation-test
IN premise — summaries/2026/08/24/park-2023-linear-representation-s5-he-was-known-as-the-warrior.md
Created 2026-08-24T17:11:03+00:00
The only concept among the 27 tested that failed the linear representation hypothesis in LLaMA-2 7B was 'thing→part', indicating not all semantic relations are linearly encoded as directions in the unembedding space Γ.
Summary
Out of 27 semantic relationships tested in LLaMA-2 7B, 26 behave like simple arrows you can slide along in the model's internal word-space, but the thing-to-part relationship (like wing pointing to bird) doesn't, meaning it is encoded in a more complex, non-directional way. This matters because any system that assumes meaning can be manipulated by vector arithmetic in that space will silently fail specifically on part-whole relations while working fine everywhere else.