fv-optimal-injection-layer-third-of-depth

IN premise — summaries/2026/08/24/todd-2023-function-vectors-s0-abstract.md

Created 2026-08-25T02:58:40+00:00

FV causal effects are maximal at early-to-middle layers (approximately |L|/3): layer 9 for GPT-J (28L), layer 15 for GPT-NeoX (44L), layer 26 for Llama-2 70B (80L), and drop sharply in late layers.

Summary

If you want to get the most causal impact out of injecting or modifying activations, aim roughly one-third of the way through the network's layers, not the front or the back. This means the final stretch of layers barely contributes to this effect, so targeting them is a waste of effort compared to hitting that early-to-middle window.