Representation Geometry

87 beliefs (87 IN, 0 OUT)

• IN bilingual-pair-filtering-requires-single-token-and-top1
  Park et al. (2023) filter bilingual translation pairs by keeping only top-1 mutual correspondences from a public word-translation dictionary and excluding pairs that are multi-token in the LLaMA-2 vocabulary (32K tokens), yielding 205–231 valid contexts per translation concept pair across four tested language pairs.
• IN causal-inner-product-captures-shared-linguistic-subconcept
  Under the causal inner product, English⇒French shows high alignment with French⇒German and French⇒Spanish (sharing the French sub-concept) but is nearly orthogonal to German⇒Spanish (no shared language); under Euclidean cosine, English⇒French appears nearly orthogonal to all other language-pair concepts.
• IN causal-inner-product-defined-as-inverse-covariance
  The causal inner product in Park et al. (2023) is defined as M = Cov(γ)⁻¹ (Eq. 3.3), where γ is the unembedding vector, and this construction enforces orthogality between causally separable concept directions.
• IN causal-inner-product-enforces-orthogality-of-independent-concepts
  The causal inner product in Park et al. (2023) is a non-Euclidean inner product with the property that concepts that can vary independently (e.g., gender vs. language) are represented as orthogonal vectors, and it induces a linear transformation that unifies the embedding space Λ and unembedding space Γ so both representations of a concept coincide.
• IN causal-inner-product-estimable-from-unembedding-matrix
  The causal inner product in Park et al. (2023) can be estimated using the LLM's unembedding matrix alone, requiring no additional gradient-based training, and its superiority over the Euclidean dot product was validated on LLaMA-2 for 27 tested concepts including language, tense, and gender.
• IN causal-inner-product-has-d-degrees-freedom
  The causal inner product is not unique: for d mutually causally separable concepts forming a basis, the metric has d degrees of freedom parameterized by a positive diagonal matrix D, with D=I_d being a canonical convention (yielding M=Cov(γ)⁻¹) rather than a theorem, and the Euclidean inner product is generally not a valid causal inner product unless M=I satisfies the constraint.
• IN causal-inner-product-outperforms-euclidean-for-concept-separability
  The LOO-estimated causal inner product strictly outperforms Euclidean cosine similarity for measuring conceptual separability in LLaMA-2's unembedding space, correctly distinguishing separable from non-separable concept pairs that cosine similarity misclassifies.
• IN causal-inner-product-strictly-superior-to-euclidean
  In LLaMA-2's unembedding space, the estimated causal inner product strictly outperforms Euclidean cosine similarity for measuring separability between concept directions, particularly for cases like frequent⇒infrequent where Euclidean overlap produces false positives.
• IN causal-separability-not-universal-among-concepts
  In Park et al. (2023), concepts are causally separable only if they can be varied freely: English⇒French and Male⇒Female are separable, but English⇒French and English⇒Russian are not (they share a source language), and the concept 'thing⇒part' is explicitly not encoded in the unembedding space Γ at all.
• IN compositionality-carnap-1947-first-explicit
  Rudolf Carnap gave the first explicit formulation of the principle of compositionality in 1947.
• IN compositionality-frege-context-principle
  Frege endorsed the context principle (meaning determined only in a whole sentence) rather than compositionality as usually stated.
• IN compositionality-logical-metonymy-example
  'John began the book' is a canonical example of logical metonymy where an event argument is inferred rather than lexically present, challenging strict compositionality.
• IN compositionality-montague-homomorphism
  In the Montague grammar tradition, compositionality is formalized as a homomorphism between an algebra of syntactic representations and an algebra of semantic objects.
• IN compositionality-partee-1984-definition
  Partee (1984) formulated compositionality as: 'The meaning of a compound expression is a function of the meanings of its parts and of the way they are syntactically combined.'
• IN compositionality-strong-vs-weak
  Strong compositionality requires meaning be determined by immediate parts and a top-level syntactic function; weak compositionality requires the complete hierarchical syntactic combination.
• IN cross-lingual-pivot-language-structure
  Cross-lingual concepts sharing a pivot language (e.g., English→French and French→German) show high causal inner product with each other, while concepts not sharing a pivot (English→French vs. German→Spanish) are nearly orthogonal under the causal inner product, validating the geometric structure of multilingual concept directions.
• IN embedding-evaluation-as-geometry-probe
  The embedding evaluation pipeline (cosine similarity → Spearman correlation, validated by both SBERT and MTEB) is not an arbitrary similarity metric but a direct linear probe of the same universal feature geometry (polytopes, hierarchical orthogonality, sparse features) revealed by internal representation analysis; the observed task-specificity in MTEB scores reflects different task-specific linear projections of this shared geometric structure rather than a fundamental failure of the metric.
• IN euclidean-inner-product-partially-works-due-to-isotropic-covariance
  The standard Euclidean inner product partially captures concept separability in LLaMA-2's unembedding space because the model's unembedding weights exhibit approximately isotropic covariance (likely from weight initialization or implicit regularization), making separable concepts roughly orthogonal, but it produces false positives (e.g., frequent→infrequent correlating with many separable concepts) that the causal inner product corrects.
• IN euclidean-orthogonality-in-llama2-is-accidental
  In LLaMA-2, causally separable concepts happen to be approximately orthogonal under the Euclidean metric due to initialization or implicit regularization favoring isotropic covariance in unembeddings, making Euclidean geometry partially functional by accident rather than reflecting genuine causal separability; the causal inner product strictly outperforms Euclidean for concept separation.
• IN fasttext-subword-character-ngram-embeddings
  fastText (Bojanowski et al., 2017) represents words via character n-grams in addition to whole-word vectors, enabling subword-level compositionality and representations for unseen or rare words in morphologically rich languages
• IN gemma-2b-tied-embeddings-invalidate-euclidean-inner-product
  In Gemma-2B, which ties unembeddings to input token embeddings, the origin of the unembedding space carries semantic weight so raw Euclidean/cosine similarity breaks down, while the causal inner product still correctly separates concept directions; in LLaMA-2, approximately isotropic unembedding covariance makes the Euclidean inner product partially functional.
• IN gemma-2b-tied-weights-break-euclidean-geometry
  Gemma-2B ties its unembedding matrix to its token embedding matrix, causing the Euclidean inner product to fail as a semantic metric for concept separation, while the causal inner product (M = Cov(γ)⁻¹) still correctly separates causally independent concepts—demonstrating that architectural choice (tied vs. untied weights) fundamentally affects whether Euclidean geometry is meaningful.
• IN intervention-context-top1-next-word-criterion
  In Park et al. (2023), intervention contexts are retained only if the target word is the top-1 next-token prediction of the model (argmax(P(next_token|context)) == target); contexts where the target is a lower-ranked candidate are discarded to ensure valid intervention measurement.
• IN intervention-scaling-monotonicity-king-to-queen
  In the Park et al. (2023) intervention experiment on context 'Long live the ', adding α·v_W (male⇒female direction) with α increasing from 0 to 0.4 shifts 'king' from rank #1 to exiting the top-5 entirely, while 'queen' reaches ranks 1–2 by α = 0.3, demonstrating monotonic linear control over concept activation in logit space.
• IN isotropic-covariance-explains-euclidean-partial-success-in-llama2
  The partial success of the Euclidean inner product in LLaMA-2 is attributed to approximately isotropic covariance of unembedding weights, likely arising from weight initialization (e.g., Xavier/Glorot) or implicit regularization during training, making unrelated directions roughly orthogonal by chance.
• IN llm-maintains-two-distinct-representation-spaces
  An LLM maintains two separate d-dimensional spaces — embedding space Λ ≃ ℝᵈ (context vectors λ(x), associated with intervention) and unembedding space Γ ≃ ℝᵈ (word vectors γ(y), associated with measurement) — and the next-token probability is P(y|x) ∝ exp(λ(x)ᵀγ(y)).
• IN llm-output-probability-couples-embedding-and-unembedding-spaces
  The LLM next-token probability is defined as P(y|x) ∝ exp(λ(x)ᵀ γ(y)) where λ(x) ∈ Λ (embedding/context space) and γ(y) ∈ Γ (unembedding/output word space), meaning the standard dot product in the softmax is a specific inner product choice that the causal inner product replaces with a structure-respecting alternative.
• IN loo-regression-estimates-unembedding-representation
  The unembedding representation used for computing causal inner products in Park et al. (2023) is estimated via a Leave-One-Out (LOO) regression procedure, connecting to broader methodology for recovering internal representations without white-box access.
• IN lrh-empirical-limit-thing-part-not-in-unembedding
  Of 27 linguistic concepts tested in LLaMA-2's unembedding space (spanning morphology, semantics, and cross-lingual mapping), the majority are linearly encoded, but some (e.g., thing⇒part) fail to show a clear signal, indicating the Linear Representation Hypothesis has empirical boundaries.
• IN lrh-three-interpretations-subspace-measurement-intervention
  The Linear Representation Hypothesis has three distinct formal interpretations: subspace (a concept is a 1-dimensional direction, e.g., γ("queen") − γ("king")), measurement (read out by a linear probe, logit-linear in the representation), and intervention (changed by adding a steering vector), with the paper proving unembedding-subspace maps to measurement and embedding-subspace maps to intervention.
• IN multi-model-geometric-convergence
  Both the polytope/orthogonality geometry (Park, validated on Gemma-2B and LLaMA-3-8B) and sparse feature structure (SAE, universal across architectures) converge on the finding that transformer representation spaces carry model-independent geometric invariants.
• IN park-2023-27-concepts-thing-part-not-encoded
  Park et al. (2023) test 27 concepts in LLaMA-2's unembedding space Γ; most are linearly encoded, but 'thing⇒part' is a named counterexample that is absent from the unembedding space, demonstrating not all concepts have linear representations.
• IN park-2023-causal-inner-product-closed-form
  The causal inner product in Park et al. (2023) has the closed-form expression ⟨γ̄, γ̄'⟩_C = γ̄^⊤ Cov(γ)^{-1} γ̄', where γ is the unembedding vector of a vocabulary-uniform word, making the inverse vocabulary-covariance matrix serve as the metric (analogous to Mahalanobis distance).
• IN park-2023-concept-ordering-determines-sign
  In Park et al. (2023), a binary concept W carries an explicit ordering (e.g., male⇒female) that determines the sign of its representation vector; reversing the order (female⇒male) negates the vector, making the sign semantically meaningful rather than arbitrary.
• IN park-2023-d-free-degrees-of-freedom
  Causal orthogonality imposes d(d−1)/2 constraints on the d×d metric matrix M, but M has d(d+1)/2 free parameters, leaving exactly d free degrees of freedom; the paper selects the diagonal D = I_d for concreteness but acknowledges non-uniqueness.
• IN park-2023-intervention-logit-space-addition
  The intervention in Park et al. (2023) is a logit-space addition (λ_{W,α}(x) = λ(x) + α·λ̄_W for α ∈ [0,1]), not an activation-space edit; at α=0.1 on context 'Long live the', top-1 shifts from 'king' to 'Queen' and 'king' drops below top-5.
• IN park-2023-riesz-map-unifies-embedding-unembedding
  Under the causal inner product, the Riesz map γ̄ ↦ ⟨γ̄, ·⟩_C sends each unembedding representation γ̄_W exactly to its embedding representation λ̄_W; in the transformed space with A = M^{1/2}, embeddings and unembeddings are literally equal (g̃_W = l̃_W) and the Euclidean inner product becomes the causal one.
• IN park-2023-softmax-affine-gauge-invariance
  The softmax training objective in LLMs is invariant under the affine transformation γ → Aγ + β, λ → A^{-⊤}λ, meaning the Euclidean inner product in the raw parameterization is not semantically meaningful and concept directions are identified only up to an invertible linear map A.
• IN park-2023-tests-27-concepts-in-llama2-unembedding
  Park et al. (2023) test 27 concepts in LLaMA-2's unembedding space, spanning multilingual translation pairs (e.g., English⇒French 'house/maison', French⇒German 'déjà/bereits') and lexical category mappings, with most encoded in Γ but some (e.g., thing⇒part) not.
• IN park-2024-70-percent-training-tokens-experiment
  In the 70% training-tokens experiment, shuffled unembeddings lose orthogonality when set inclusion is violated, while original unembeddings retain orthogonality, evidencing genuine semantic encoding.
• IN park-2024-causal-inner-product-whitening
  The hierarchical orthogonality in Park et al. 2024 is only visible under a 'causal inner product' that includes both centering and whitening via an invertible matrix A; under the raw Euclidean inner product (centering only), the orthogonality breaks down.
• IN park-2024-inner-product-zero-high-probability
  The algebraic derivation in Eqs. G.3–G.5 shows the inner product between the child–parent difference vector and the parent vector equals 0 with high probability as sequence length grows.
• IN park-2024-lda-estimator-preferred
  The LDA-based direction estimator (eq. 5.2) is preferred over the simple mean vector E(g_w) for estimating ℓ̄_w due to lower variance.
• IN park-2024-llama3-hierarchy-orthogonality
  In LLaMA-3's representation space, the vector (child − parent) is approximately orthogonal to the parent vector for WordNet taxonomic relations, with cosine similarity approaching zero.
• IN park-2024-orthogonality-noun-verb
  The hierarchical orthogonality property in LLaMA-3 holds across both WordNet noun and verb hierarchies.
• IN park-2024-proposition-9-simplices
  Proposition 9 states that if k mutually exclusive attributes can independently have arbitrary joint distributions, their vector representations form a (k−1)-simplex.
• IN park-2024-shuffled-baseline-collapses-orthogonality
  When WordNet parent/child assignments are randomly shuffled in the unembedding, the predicted orthogonality structure collapses (cosine similarity departs from zero).
• IN park-2024-theorem-8-four-conditions
  Theorem 8 in Park et al. 2024 specifies four distinct orthogonality conditions (a–d) covering parent⊥child-parent, parent⊥sibling-contrast, parent-contrast⊥child-contrast, and three-level chain orthogonality.
• IN park-2024-token-space-prefix-convention
  In Park et al. 2024, tokens are prefixed with a space character to represent full-word next-token prediction, and both plural/singular and upper/lower-case variants are included.
• IN park-2024-validated-gemma2b-llama3-8b
  The geometric structure results were empirically validated on both Gemma-2B and LLaMA-3-8B using WordNet hierarchies.
• IN park-2025-binary-contrast-vector-difference
  Corollary 6 establishes that binary contrasts are vector differences of binary features: ℓ̄_{w0⇒w1} = ℓ̄_{w1} − ℓ̄_{w0}, enabled by representations having well-defined magnitudes (not just directions).
• IN park-2025-causal-inner-product-whitening-formula
  The Causal Inner Product is constructed by whitening the unembedding space: g(y) = Cov(γ)^(−1/2) · (γ(y) − E[γ]), after which the Euclidean inner product equals the causal inner product.
• IN park-2025-centering-alone-insufficient
  Centering unembedding vectors alone (preserving the Euclidean inner product) does not yield hierarchical orthogonality; both centering and whitening (covariance inversion) are required for the Causal Inner Product.
• IN park-2025-code-repository
  The code for the paper is available at github.com/KihoPark/LLM_Categorical_Hierarchical_Representations.
• IN park-2025-colinearity-binary-features
  Binary feature vectors for hierarchical concepts (e.g., 'animal,' 'mammal,' 'bird') are strongly co-linear in the standard representation space, making them difficult to disentangle via sparse autoencoder approaches.
• IN park-2025-direct-sum-space-decomposition
  The combination of polytope representations and hierarchical orthogonality (Theorem 8) implies the full representation space decomposes as a direct sum of orthogonal subspaces, one per level of the hierarchy.
• IN park-2025-empirical-controls-shuffle-random
  Three empirical controls rule out artifacts: (1) shuffling unembeddings destroys the structure, (2) random parent selection yields non-zero cosine (ruling out high-dim geometry), (3) independent 70% train splits break set-inclusion orthogonality only in shuffled embeddings.
• IN park-2025-funding-onr-open-philanthropy
  The paper was funded by U.S. Office of Naval Research grant N00014-23-1-2591 and Open Philanthropy.
• IN park-2025-gemma-2b-specifications
  Gemma-2B has 2 billion parameters, was pretrained on 3 trillion tokens, has a 256K vocabulary, and uses a 2048-dimensional representation space.
• IN park-2025-hierarchical-orthogonality-theorem-8
  Theorem 8 states that for subordinate concepts z ≺ w, the parent vector is orthogonal to the child-minus-parent residual: ℓ̄_w ⊥ (ℓ̄_z − ℓ̄_w), and sibling contrast vectors are mutually orthogonal.
• IN park-2025-iclr-categorical-polytope-geometry
  Park et al. (ICLR 2025) prove that categorical concepts in LLM representation spaces are geometrically represented as polytopes (convex hulls of vertex vectors), with 'natural' concepts forming (k−1)-simplices.
• IN park-2025-lda-estimator-lower-variance
  The LDA direction estimator (Eq. 5.2, using Ledoit-Wolf shrinkage for covariance) is more stable (lower variance) than the simple mean vector E(g_w) for estimating ℓ̄_w.
• IN park-2025-natural-concepts-simplices
  Proposition 9 states that 'natural' categorical concepts (where the model can freely manipulate any output marginal) are represented as (k−1)-simplices rather than general polytopes.
• IN park-2025-no-hyperbolic-geometry-found
  The paper reports a negative result: no hyperbolic geometric structure was detected in LLM representation spaces, despite prior work mapping semantic hierarchy to hyperbolic embeddings.
• IN park-2025-norm-additivity-orthogonality-proof
  The orthogonality in Theorem 8(d) for consecutive hierarchy levels is proven via the parallelogram/norm-additivity condition: ‖ℓ̄_{w₁}−ℓ̄_{w₀}‖² + ‖ℓ̄_{w₂}−ℓ̄_{w₁}‖² = ‖ℓ̄_{w₂}−ℓ̄_{w₀}‖².
• IN park-2025-set-inclusion-spurious-orthogonality
  Shuffled unembeddings reproduce orthogonality between child-parent and parent vectors purely from set inclusion (Y(z) ⊆ Y(w)), not semantic meaning, demonstrating that set inclusion alone is insufficient to explain the geometric structure.
• IN park-2025-softmax-invariance-whitening
  The whitening/space-unification transformation A(γ(y) − γ̄₀) with corresponding λ̃(x) = A^(−⊤) λ(x) does not change softmax output probabilities P(y|x).
• IN park-2025-theorem-4-magnitude-binary-projection
  Theorem 4 states that for a binary feature w with linear representation ℓ̄_W, ℓ̄_Wᵀ g(y) = b_w (a positive constant) for all y ∈ Y(w) and ℓ̄_Wᵀ g(y) = 0 for y ∉ Y(w).
• IN park-2025-validation-models-wordnet
  The results were empirically validated on Gemma-2B and LLaMA-3-8B using 593 noun and 364 verb WordNet synsets (retained if containing ≥50 words in the model vocabulary).
• IN park-2025-venue-iclr-2025
  The paper 'Geometry of Categorical and Hierarchical Concepts' was published as a conference paper at ICLR 2025 (International Conference on Learning Representations).
• IN park-2025-whitening-final-layer-only
  The whitening transformation used to define the canonical representation space applies only to the final layer's unembedding matrix, leaving internal-layer geometry as an open problem.
• IN park2025-binary-directions-features-vectors-categorical-polytopes
  Park et al. (ICLR 2025) formalizes binary concepts with natural contrasts as directions, features without contrasts as vectors, and categorical concepts as polytopes in LLM representation space.
• IN park2025-code-repository
  The code for Park et al. (2025) is available at github.com/KihoPark/LLM_Categorical_Hierarchical_Representations.
• IN park2025-final-layer-final-token
  LLM representations for concept geometry in Park et al. (2025) are read at the final layer, final token position of the transformer.
• IN park2025-validation-models-concepts
  Park et al. (2025) validated their geometric representation results on Gemma and LLaMA-3 using 900+ hierarchically related concepts from WordNet.
• IN park2025-wordnet-hierarchy-source
  WordNet (Miller, 1995) is used as the source of hierarchical concept relationships for empirical validation in Park et al. (2025).
• IN residual-stream-universal-substrate-v2
  SAE targets the middle-layer residual stream, ROME edits the mid-layer FFN value projection (W_V), and Park's whitening applies only to the final-layer unembedding matrix. Each approach selects a specific depth as its intervention or representation target, but they do not jointly identify the residual stream as the primary locus of interpretable geometric structure; in particular, Park explicitly leaves internal-layer geometry as an open problem.
• IN riesz-isomorphism-unifies-embedding-and-unembedding-spaces
  Under the causal inner product, the Riesz isomorphism (Theorem 3.2) maps each unembedding representation γ̄_W to its corresponding embedding representation λ̄_W via ⟨γ̄_W, ·⟩_C = λ̄_Wᵀ, collapsing the two separate representation spaces into a single unified geometric framework.
• IN riesz-map-as-unified-operational-framework-v2
  The Riesz map under the causal (whitened) inner product serves as a unifying canonical isomorphism that both (a) renders embedding and unembedding semantically equivalent as a well-defined algebraic object in a model-independent vector space, and (b) provides a coherent geometric translation between ROME's key (MLP input activation) and value (MLP output projection) representations, reframing what appear as two independent weight matrices as a single dual-geometric object. In this sense, the Riesz map offers a shared algebraic framework under which semantic measurement and parametric editing can be understood as related views of one underlying structure, rather than as fully independent mechanisms.
• IN riesz-map-unifies-full-logical-structure-v2
  Within the framework of over-complete superposition in a shared substrate, the Riesz map under the causal (whitened) inner product serves as the unique canonical isomorphism identifying embedding and unembedding as semantically equivalent objects in a model-independent vector space, while the same structural premise entails that cross-model geometric convergence is a logical necessity rather than a contingent coincidence, and that the semantic space admits a decomposition into categorical polytope subspaces and hierarchical orthogonality subspaces—unifying three related observations under a single algebraic account of how discrete and graded meaning coexist in an over-complete representation.
• IN sae-functional-abstraction-extends-geometric-scope-v2
  SAE features activating on functional analogies (a transit feature firing on wormholes alongside physical transit entities) and on cross-modal inputs (text-trained features responding to image inputs) suggest that LLM representation spaces capture abstract, generalized concepts and shared cross-modal structure alongside the categorical polytope geometry described by Park et al., pointing toward a representational picture that extends beyond strict category membership to include non-literal generalization.
• IN shared-language-causal-entanglement-structure
  Language-translation concepts sharing a common language (e.g., English⇒French and French⇒German) show high causal inner product with each other, while non-overlapping pairs (German⇒Spanish) are nearly orthogonal, confirming geometric structure mirrors linguistic compositional relationships.
• IN shared-language-pivot-affects-cross-lingual-concept-geometry
  Under the causal inner product in LLaMA-2, the English→French concept has high inner product with French→German and French→Spanish (sharing 'French' as a pivot language) but remains nearly orthogonal to German→Spanish, while under the Euclidean inner product English→French has low inner product with other language concepts.
• IN superposition-necessitates-covariance-whitening-v2
  Over-complete superposition in the residual stream (identified as a locus of interpretable structure across multiple depths) is a primary structural condition that motivates the use of empirical second-moment matrices as a natural inner product for reasoning about individual features and performing targeted interventions; independent lines of work (ROME's key-space projection, Park's unembedding whitening) converge on this covariance-geometry approach as a practical framework for feature manipulation in over-complete representation spaces.
• IN thing-to-part-failed-linear-representation-test
  The only concept among the 27 tested that failed the linear representation hypothesis in LLaMA-2 7B was 'thing→part', indicating not all semantic relations are linearly encoded as directions in the unembedding space Γ.
• IN training-identifies-representations-only-up-to-affine-transform
  Training an LLM identifies the embedding and unembedding representations only up to an invertible affine transformation (γ(y) → Aγ(y) + β, λ(x) → A⁻ᵀλ(x)), which preserves all softmax probabilities, meaning no canonical Euclidean inner product is selected by training alone.
• IN weight-tying-shares-embedding-unembedding
  Weight tying uses the transpose of the embedding matrix as the un-embedding matrix, halving embedding parameters and preventing training divergence.