park-2023-tests-27-concepts-in-llama2-unembedding

IN premise — summaries/2026/08/24/park-2023-linear-representation-s14-the-prince-matured-and-eventually-became-the.md

Created 2026-08-24T17:11:02+00:00

Park et al. (2023) test 27 concepts in LLaMA-2's unembedding space, spanning multilingual translation pairs (e.g., English⇒French 'house/maison', French⇒German 'déjà/bereits') and lexical category mappings, with most encoded in Γ but some (e.g., thing⇒part) not.

Summary

Park et al. empirically showed that LLaMA-2's internal geometry can represent most cross-language translation relationships and word-category pairings as discoverable directions in its output layer, but a few relationships (like the thing-to-part mapping) fall outside that structure. This matters because it tells us the model's representational space is broad but incomplete, setting a baseline for what downstream reasoning or editing methods can reliably operate on.