rome-covariance-from-wikipedia
IN premise — summaries/2026/08/24/meng-2022-rome-s2-interventions-on-activations-for-tracing-information-flow.md
Created 2026-08-25T02:58:14+00:00
In ROME, the key covariance matrix C = KKᵀ is pre-computed from a Wikipedia text sample and is not recomputed per edit.
Summary
ROME locks in its understanding of "normal" text by computing a statistical summary from a fixed Wikipedia sample once, then reuses that same summary for every edit it makes. This keeps each edit fast and cheap, but it also means the system never updates its notion of what typical knowledge looks like, even as many edits pile up and change the model.