partial-whitening-cost-geometric-necessity
OUT derived (depth 6)
Created 2026-08-25T03:48:00+00:00 · Reviewed 2026-08-25T04:02:18+00:00
The O(D²) cost of a rank-one ROME edit is the exact geometric price of single-direction partial whitening: modifying one row/column of the inverse covariance while preserving D−1 others is irreducibly O(D²) because C⁻¹ is a full D×D matrix, making the complexity bound a theorem of the geometry rather than an engineering limitation.
Justifications
SL — The partial-whitening interpretation (d3) explains *why* the cost is D²; the subspace boundary (d5) explains *why* only one direction is modified; the necessity claim (d3) confirms it is a lower bound. Together they show the geometry *is* the cost function.
Antecedents (all must be IN):
- OUT edit-complexity-is-geometrically-necessary — The O(D²) cost of a rank-one knowledge edit is a fundamental lower bound imposed by the geometry of superposition, not an implementation artifact: any edit that preserves the covariance-geometric structure of the residual stream must operate in the whitened D-dimensional subspace, incurring at least O(D²) parameter modification
- OUT rome-edit-as-partial-whitening — A rank-one ROME weight update is operationally equivalent to a local, single-direction whitening of the residual stream: the C⁻¹k* projection in the update formula performs precisely the covariance-normalization that superposition necessitates, but confined to one key direction—making each edit a "partial whitening" that corrects one superposed feature without disturbing the orthogonal complement.
- OUT parametric-write-subspace-boundary — The operational boundary between parametric recall and contextual retrieval is precisely the geometric boundary of the rank-one addressable subspace: facts whose subject-key projection aligns with the locally-stored key covariance are parametrically editable, while facts outside this subspace must be externally supplied.
Dependents
These beliefs depend on this one:
- OUT multi-hop-drop-as-rank-one-boundary-signature — The dramatic 40.5%→7.4% multi-hop accuracy collapse after ROME editing is the operational fingerprint of the rank-one write boundary: multi-hop reasoning requires *composition* of multiple whitened directions, but the covariance-whitened subspace structurally supports only single-direction writes, making multi-hop parametric recall geometrically unreachable.