edit-complexity-is-geometrically-necessary
OUT derived (depth 3)
Created 2026-08-25T03:13:05+00:00 · Reviewed 2026-08-25T04:02:18+00:00
The O(D²) cost of a rank-one knowledge edit is a fundamental lower bound imposed by the geometry of superposition, not an implementation artifact: any edit that preserves the covariance-geometric structure of the residual stream must operate in the whitened D-dimensional subspace, incurring at least O(D²) parameter modification
Justifications
SL — rome-edit-complexity-o-d2 states the O(D²) cost empirically; superposition-necessitates-covariance-whitening (depth-2) establishes that whitening is a structural necessity of over-complete representation. Together: the cost is not a design choice of ROME but a geometric invariant of any consistent edit in a superposed system.
Antecedents (all must be IN):
- IN rome-edit-complexity-o-d2 — ROME's rank-one edit is O(D²) per edit (a single outer product Λ(C⁻¹k*)ᵀ) rather than O(params), avoiding full retraining.
- OUT superposition-necessitates-covariance-whitening — Over-complete superposition in the shared residual-stream substrate is the precise structural condition that necessitates covariance/whitening (second-moment projection) as the canonical tool for isolating individual features and performing targeted rank-one edits; without superposition, raw Euclidean geometry would suffice and the entire covariance-geometry framework would be unnecessary.
Dependents
These beliefs depend on this one:
- OUT partial-whitening-cost-geometric-necessity — The O(D²) cost of a rank-one ROME edit is the exact geometric price of single-direction partial whitening: modifying one row/column of the inverse covariance while preserving D−1 others is irreducibly O(D²) because C⁻¹ is a full D×D matrix, making the complexity bound a theorem of the geometry rather than an engineering limitation.