park2025-orthogonality-causally-separable
IN premise — summaries/2026/08/24/park-2024-categorical-hierarchical-concepts-s0-abstract.md
Created 2026-08-25T02:58:24+00:00
Causally separable binary concepts (independently manipulable) are represented by orthogonal directions in LLM representation space under a suitable inner product.
Summary
When two on-off concepts in an LLM can be independently flipped without affecting each other, the model's internal math places their directions at a right angle to one another. This matters because it turns geometric measurement into a reliable test for genuine independence, meaning an intervention on one concept won't silently bleed into the other and the model's geometry faithfully mirrors the causal structure it encodes.