fv-vocabulary-reconstruction-fails

IN premise — summaries/2026/08/24/todd-2023-function-vectors-sR-references.md

Created 2026-08-25T02:58:42+00:00

Reconstructing a vector that decodes to the same output-word distribution (near-zero KL divergence) as the FV does not recover FV task performance, proving FVs carry information beyond a simple vocabulary-boosting offset.

Summary

Creating a vector that tricks the model into producing the same word choices as an FV still doesn't let it do the FV's actual job, showing that FVs carry task-specific structure that goes well beyond steering vocabulary. In practice, this means you can't substitute a simpler "word-boosting" shortcut for a real FV — the functional information encoded in it is irreducible.