fv-late-layer-near-zero-effect

IN premise — summaries/2026/08/24/todd-2023-function-vectors-sR-references.md

Created 2026-08-25T02:58:42+00:00

Function vectors exert near-zero causal effect when added at late layers (e.g., layers 90%+ of depth), indicating they trigger computations through intermediate nonlinear feed-forward layers rather than acting on output logits.

Summary

Function vectors only work when injected early or mid-network, because they act as instructions to the model's internal processing stages rather than as a final nudge to the output. This means the system's behavior is shaped by how intermediate layers compute and transform information, not by simply shifting the answer at the end, so any intervention or alignment strategy needs to engage the model's reasoning machinery rather than just steering the result.