park2025-binary-directions-features-vectors-categorical-polytopes

IN premise — summaries/2026/08/24/park-2024-categorical-hierarchical-concepts-s0-abstract.md

Created 2026-08-25T02:58:23+00:00

Park et al. (ICLR 2025) formalizes binary concepts with natural contrasts as directions, features without contrasts as vectors, and categorical concepts as polytopes in LLM representation space.

Summary

This gives us a geometric grammar for how large language models encode different kinds of concepts: binary oppositions (like hot/cold) live along directional axes, standalone attributes sit at fixed vector offsets, and multi-category groupings (like days of the week) form the vertices of a geometric shape. It matters because it turns the fuzzy idea of "concept space" into something you can actually measure, manipulate, and reason about with linear algebra, giving a shared vocabulary for how concepts of different types relate to one another inside a model.