rome-c-gram-matrix-definition

IN premise — summaries/2026/08/24/meng-2022-rome-sR-references-chunk-1.md

Created 2026-08-25T02:58:16+00:00

In ROME's derivation, C = KKᵀ is the Gram matrix of the key set, and its inverse C⁻¹ projects the new key into the subspace orthogonal to existing keys to minimize disruption.

Summary

ROME uses the inverse of the key-similarity matrix (the Gram matrix) to steer a newly inserted fact into a direction that doesn't overlap with facts the model already stores. In practice, this is the mathematical guarantee that editing one fact won't accidentally overwrite or distort the others, which is why the edit stays surgical rather than global.

Dependents

These beliefs depend on this one:

IN covariance-geometry-as-canonical-tool — Independent lines of work (ROME's key-space projection and Park's unembedding whitening) converge on using empirical second-moment matrices to define the "correct" inner product for reasoning about transformer representations.