fv-alternative-multilayer-injection-formula

IN premise — summaries/2026/08/24/todd-2023-function-vectors-s2-function-vector-causal-effects-cannot-be-recovered-from-the--chunk-3.md

Created 2026-08-25T02:58:41+00:00

The alternative multi-layer FV injection is defined as h'_k = h_k + Σ_{(ℓ,j)∈A, ℓ=k} ā^t_ℓj, adding each head's activation to its own layer rather than summing all into one layer.

Summary

This defines how fine-tuning adjustments get blended back into a transformer model: each adjustment is added only to the layer it was learned from, instead of dumping everything into one shared layer. The practical upshot is that the model keeps its layer-by-layer structure intact, so the learned corrections don't bleed into or overwrite each other across layers.