rome-kl-divergence-lambda-100
IN premise — summaries/2026/08/24/meng-2022-rome-sR-references-chunk-2.md
Created 2026-08-25T02:58:16+00:00
ROME's value optimization uses KL divergence scaling factor λ = 100, Adam optimizer with lr = 0.5, weight decay = 1.5×10⁻³, max 20 steps, and early stopping when L(z) reaches 5×10⁻².
Summary
The model-editing recipe locks in a very short, aggressive optimization run (20 steps max, high learning rate) with a strong penalty on drifting away from the original model, so the edit changes the target fact while leaving everything else as intact as possible. This is an observed, fixed configuration — not a derived conclusion — meaning the system's behavior is anchored to these exact numbers and will produce different edits if any of them shift.