rome-edit-complexity-o-d2

IN premise — summaries/2026/08/24/meng-2022-rome-sR-references.md

Created 2026-08-25T02:58:17+00:00

ROME's rank-one edit is O(D²) per edit (a single outer product Λ(C⁻¹k*)ᵀ) rather than O(params), avoiding full retraining.

Summary

ROME rewrites a fact in the model by adjusting only a single D-by-D block of weights through a simple vector outer product, so each edit costs roughly D squared operations rather than scaling with the total parameter count. In practice, this means the system can correct a specific piece of knowledge quickly and cheaply without paying the full cost of retraining the network.

Dependents

These beliefs depend on this one:

OUT edit-complexity-is-geometrically-necessary — The O(D²) cost of a rank-one knowledge edit is a fundamental lower bound imposed by the geometry of superposition, not an implementation artifact: any edit that preserves the covariance-geometric structure of the residual stream must operate in the whitened D-dimensional subspace, incurring at least O(D²) parameter modification