fv-injection-layer-selection-per-model

IN premise — summaries/2026/08/24/todd-2023-function-vectors-s2-function-vector-causal-effects-cannot-be-recovered-from-the-.md

Created 2026-08-25T02:58:41+00:00

FV injection layer selection (≈ |L|/3) is: GPT-J layer 9, GPT-NeoX layer 15, Llama 2 7B layer 11, Llama 2 13B layer 14, Llama 2 70B layer 26.

Summary

This is a practical lookup table that tells the system exactly which layer to insert feature-vector representations into for each model variant, following a rough one-third-depth rule. It matters because injecting at the wrong layer would either interfere with early token processing or miss the stage where representations are abstract enough to be useful, so the system relies on these specific numbers to get correct behavior across GPT-J, GPT-NeoX, and the three Llama 2 sizes.