lrh-validated-on-llama2-7b-27-concepts

IN premise — summaries/2026/08/24/park-2023-linear-representation-s4-experiments.md

Created 2026-08-24T17:11:03+00:00

The Linear Representation Hypothesis was empirically validated on LLaMA-2 7B (decoder-only Transformer, 32K vocabulary) across 27 concepts comprising 22 from BATS 3.0, 4 language pairs (EN→FR, FR→DE, FR→ES, DE→ES), and 1 frequency pair, with code at github.com/KihoPark/linear_rep_geometry.

Summary

The idea that concepts live at specific directions inside a language model's embedding space was actually tested and confirmed on LLaMA-2 7B across 27 different concepts, including real vocabulary items, cross-language translation pairs, and frequency distinctions. This matters because it gives the system empirical grounds to treat vector geometry in that model as a reliable proxy for conceptual structure, rather than a loose analogy.