geometric-editing-addressability-bound
OUT derived (depth 3)
Created 2026-08-25T03:03:22+00:00 · Reviewed 2026-08-25T04:02:18+00:00
The covariance-geometry framework defines a precise and minimal addressable space for knowledge editing (rank-one updates to a single MLP value projection), but the combination of superposition and distributed corpus acquisition structurally bounds this to single-fact local corrections—edits cannot create novel multi-hop associations because the target knowledge was never locally consolidated in the first place.
Justifications
SL — Covariance geometry defines the editing mechanism; distributed acquisition explains why the target may not be locally consolidated; superposition explains why perturbing one superposed feature necessarily bleeds into others—together they form a complete explanation of why editing is bounded
Antecedents (all must be IN):
- OUT covariance-geometry-unifies-analysis-and-editing — The mathematically principled framework for both interpreting (SAE feature extraction, Park polytope analysis) and modifying (ROME rank-one edits) LLM representations is second-moment covariance geometry applied to the residual stream, since C = KKᵀ whitening defines the canonical coordinate system in which all three operations become linear algebra on the same substrate.
- OUT distributed-acquisition-bounds-editing-scope — Knowledge editing is fundamentally a correction mechanism (fixing mis-encoded local associations) rather than a creation mechanism, because novel factual knowledge requires the distributed corpus exposure that a single rank-one update on one MLP layer cannot replicate.
- OUT superposition-as-compositional-basis — Superposition is the fundamental compositional mechanism in LLMs: the 10–200× over-complete expansion (SAE), the key-value memory structure (ROME's W_fc/W_proj), and the direct-sum space decomposition (Park's polytope+orthogonality) are three independent geometric consequences of the same over-completeness.
Dependents
These beliefs depend on this one:
- OUT context-window-ephemeral-edit — The long-context window with near-perfect in-context key-value retrieval functions as an ephemeral, weight-free knowledge editing mechanism: injecting a fact into the 200K-token context is functionally equivalent to a rank-one edit that bypasses the parametric write subspace, providing a complementary write channel with zero persistence cost.
- OUT editable-semantic-space — The residual stream, equipped with its covariance-geometric structure, constitutes a well-defined editable semantic space in which knowledge can be read (SAE feature activations, Park polytope coordinates) and written (ROME rank-one value-projection updates) as addressable, independently manipulable units.
- OUT editing-reliability-under-superposition — Rank-one knowledge editing is reliable as a single-fact correction mechanism in superposed models because covariance whitening provides sufficient feature separation to isolate the target association from the background superposition, making the edit direction well-defined and locally confined.
- OUT evaluation-geometry-predicts-editability — The convergence of evaluation geometry (cosine/Spearman in SBERT/MTEB) and editing geometry (covariance whitening in ROME) on the same second-moment structure means that improving evaluation alignment and enabling reliable editing are two operational views of the same geometric optimization over the residual-stream covariance.
- OUT head-tail-geometric-divide — The parametric/contextual knowledge split is geometrically grounded rather than merely frequency-driven: head-of-distribution facts occupy individually addressable directions in the covariance-whitened feature space (enabling rank-one editing), while long-tail facts are distributed across superposed features where no single direction isolates the knowledge, making external retrieval the only faithful access mechanism.
- IN linearity-boundary-of-geometric-framework — The geometric framework's explanatory boundary is precisely the boundary of linearity: ROME cannot edit non-factual (logical/spatial/numerical) associations because they are not rank-one key-value pairs, and Park's framework cannot encode the 'thing⇒part' relation because it is a non-linear constraint in the unembedding space—both failures share the root cause of the linear-subspace assumption.
- OUT parametric-write-subspace-boundary — The operational boundary between parametric recall and contextual retrieval is precisely the geometric boundary of the rank-one addressable subspace: facts whose subject-key projection aligns with the locally-stored key covariance are parametrically editable, while facts outside this subspace must be externally supplied.
- OUT read-write-asymmetry — A structural asymmetry governs LLM knowledge: the read channel (feature-level interpretation via SAE, Park polytope analysis, embedding evaluation) operates in a rich, model-independent geometric space, while the write channel (editing) is impoverished to parametric rank-one MLP updates in a single layer—superposition enriches the feature space for reading but makes feature-level writing intractable.
- OUT superposition-covariance-editability-triangle — Superposition, covariance whitening, and rank-one editability form a closed logical triangle in which each property necessitates the others: over-complete superposition requires covariance separation for feature addressability, covariance separation defines the geometric space in which rank-one updates are well-defined, and the boundedness of rank-one editing confirms the addressable space is finite.