linearity-boundary-of-geometric-framework
IN derived (depth 4)
Created 2026-08-25T03:50:14+00:00 · Reviewed 2026-08-25T04:28:09+00:00
The geometric framework's explanatory boundary is precisely the boundary of linearity: ROME cannot edit non-factual (logical/spatial/numerical) associations because they are not rank-one key-value pairs, and Park's framework cannot encode the 'thing⇒part' relation because it is a non-linear constraint in the unembedding space—both failures share the root cause of the linear-subspace assumption.
Summary
There is a hard ceiling on what geometric and linear editing methods can reach inside a language model: they work for simple "this is that" facts, but they structurally cannot touch relationships like "this part belongs to that thing," logical inferences, or spatial reasoning, because those don't live as simple directional points in the model's output space. This means any system built on the assumption that knowledge is stored as linear key-value directions will silently fail on a large and important class of knowledge, and diagnosing that failure requires checking whether the target relationship is even linearly representable before attempting an edit.
Justifications
This belief has 3 justifications — it is IN if any one holds.
SL — Two independent empirical failures (ROME's scope limitation to entity-relation-object, Park's failure on 'thing⇒part') converge on the same geometric boundary: the covariance-whitened space is a linear subspace, and any relation that is intrinsically non-linear (composition, part-whole, logical entailment) falls outside its addressable range. Each antecedent independently demonstrates the boundary.
Antecedents (all must be IN):
- IN rome-scope-limitation — ROME and Causal Tracing address factual (entity-relation-object) associations only; logical, spatial, and numerical knowledge are explicitly out of scope.
SL — Two independent empirical failures (ROME's scope limitation to entity-relation-object, Park's failure on 'thing⇒part') converge on the same geometric boundary: the covariance-whitened space is a linear subspace, and any relation that is intrinsically non-linear (composition, part-whole, logical entailment) falls outside its addressable range. Each antecedent independently demonstrates the boundary.
Antecedents (all must be IN):
- IN park-2023-27-concepts-thing-part-not-encoded — Park et al. (2023) test 27 concepts in LLaMA-2's unembedding space Γ; most are linearly encoded, but 'thing⇒part' is a named counterexample that is absent from the unembedding space, demonstrating not all concepts have linear representations.
SL — Two independent empirical failures (ROME's scope limitation to entity-relation-object, Park's failure on 'thing⇒part') converge on the same geometric boundary: the covariance-whitened space is a linear subspace, and any relation that is intrinsically non-linear (composition, part-whole, logical entailment) falls outside its addressable range. Each antecedent independently demonstrates the boundary.
Antecedents (all must be IN):
- OUT geometric-editing-addressability-bound — The covariance-geometry framework defines a precise and minimal addressable space for knowledge editing (rank-one updates to a single MLP value projection), but the combination of superposition and distributed corpus acquisition structurally bounds this to single-fact local corrections—edits cannot create novel multi-hop associations because the target knowledge was never locally consolidated in the first place.