memit-targets-mlp-layers-not-attention
IN premise — summaries/2026/08/24/meng-2022-memit-s0-abstract.md
Created 2026-08-25T02:58:13+00:00
MEMIT targets MLP (feed-forward) layers specifically as key-value memory stores, not attention layers, for factual knowledge editing
Summary
Factual knowledge in the model lives in the feed-forward (MLP) blocks as lookup tables, not in the attention layers that govern how tokens relate to each other. This matters because it means a knowledge edit can surgically change what the model "remembers" without disturbing its general language-flow patterns, keeping the rest of generation stable after the update.