fv-decoded-vocabulary-insufficiency

IN premise — summaries/2026/08/24/todd-2023-function-vectors-s0-abstract-chunk-1.md

Created 2026-08-25T02:58:40+00:00

Reconstructing a Function Vector from its top decoded tokens (even all 50k vocabulary tokens) yields significantly lower task-execution accuracy, indicating FVs carry information beyond their surface decoded vocabulary.

Summary

A Function Vector holds meaning that you simply cannot recover by looking at which words it most strongly points to, even if you examine the full 50k-token vocabulary. This means the system has to treat these vectors as rich, high-dimensional objects rather than reducible word lists, because any workflow that tries to flatten them into surface tokens will silently lose capability.