rome-scope-limitation
IN premise — summaries/2026/08/24/meng-2022-rome-s4-related-work.md
Created 2026-08-25T02:58:15+00:00
ROME and Causal Tracing address factual (entity-relation-object) associations only; logical, spatial, and numerical knowledge are explicitly out of scope.
Summary
ROME and Causal Tracing can only patch simple "who-did-what-to-whom" style facts inside a model. That means if you need to fix a logical rule, a spatial relationship, or a numerical computation, neither of these tools will help — the system must rely on some other mechanism for those kinds of changes.
Dependents
These beliefs depend on this one:
- OUT knowledge-lifecycle-closed-loop — The full knowledge lifecycle in LLMs—distributed corpus acquisition → local MLP key-value storage → parametric recall → rank-one editing correction—forms a closed loop unified by the covariance geometry, where each stage is addressable in the same second-moment coordinate space.
- IN linearity-boundary-of-geometric-framework — The geometric framework's explanatory boundary is precisely the boundary of linearity: ROME cannot edit non-factual (logical/spatial/numerical) associations because they are not rank-one key-value pairs, and Park's framework cannot encode the 'thing⇒part' relation because it is a non-linear constraint in the unembedding space—both failures share the root cause of the linear-subspace assumption.
- OUT local-mlp-editing-principle — The MLP-as-key-value-memory structure provides a principled, minimal-intervention editing mechanism for locally-stored factual (entity-relation-object) associations, with the edit's efficacy and specificity guaranteed by the rank-one update's geometric isolation in the covariance-projected key space.