fv-merullo-contrast-late-vs-mid-layer
IN premise — summaries/2026/08/24/todd-2023-function-vectors-s2-function-vector-causal-effects-cannot-be-recovered-from-the--chunk-1.md
Created 2026-08-25T02:58:40+00:00
Merullo et al. (2023) describe vector offsets most effective at late layers (resembling embedding offsets), whereas FVs show the opposite pattern with peak causal effects at early/mid layers, establishing a mechanistic distinction.
Summary
Two popular ways of nudging a neural network's internal representations end up targeting very different stages of processing: one works best near the output (much like adjusting the final lookup table), while the other has its strongest effect early to mid in the computation. This means they are not interchangeable tools; they probe and modify different computational steps, so picking the right one depends on which layer of the network's reasoning you actually want to influence.