Function Vectors

70 beliefs (70 IN, 0 OUT)

The topic "Function Vectors" encompasses a family of results showing that transformer models encode discrete task-execution functions as compact vector quantities in their activation spaces, and that these vectors can be extracted, injected, composed, and even translated across incompatible embedding spaces. Three research threads contribute to this picture. Todd et al., published at ICLR 2024 (fv-iclr2024-publication), define Function Vectors (FVs) as sums of mean task-conditioned activations over a small set of attention heads identified via causal mediation analysis, with causal effects peaking when the vector is injected at roughly one-third of model depth (fvs-constructed-from-middle-layer-attention-heads, fv-optimal-injection-layer-third-of-depth). Hendel et al. (2023) independently show that a single hidden-state vector at an intermediate layer, extracted from the separator token of in-context demonstrations, suffices to drive task execution across 18 tasks in four categories, establishing what they call a "task vector" in activation space as distinct from the weight-space task arithmetic of Ilharco et al. (hendel-task-arithmetic-ilharco-2023, hendel-2023-task-vector-activation-space). Finally, Jha et al. at NeurIPS 2025 (vec2vec-published-neurips-2025-arxiv) push the idea further by demonstrating that functional information encoded in one embedding space can be translated into an entirely incompatible space without paired data or encoder access, treating vectors as carriers of transformable semantic content.

The Todd et al. FV results are the most mechanistically detailed. Extraction selects heads by ranking average indirect effect over 25 corrupted 10-shot prompts, then averages task-conditioned activations over 100 clean 10-shot prompts (fv-extraction-uses-100-clean-prompts, fv-extraction-head-count-gptj). The number of heads in the FV-transporting set scales roughly proportionally to model size, from 10 heads in GPT-J to 100 in Llama 2 70B (fv-head-count-scales-with-model-size, fv-head-set-sizes-by-model), while the maximum per-head AIE actually decreases in larger models (fv-max-aie-decreases-with-model-size), suggesting larger models distribute the same function across more heads. High-AIE heads cluster in middle layers across GPT-J, Llama 2, and GPT-NeoX, with some overlap with known induction heads (fv-gptneoxx-middle-layer-clustering, fv-top10-gptj-induction-heads). A key negative result is that FVs are not merely vocabulary-boosting offsets: reconstructing a vector from its top 50k decoded tokens yields poor task accuracy (fv-decoded-vocabulary-insufficiency), and a vector matching the FV's output distribution in near-zero KL divergence still fails to recover task performance (fv-vocabulary-reconstruction-fails). A related impossibility result shows no constant offset can implement a cyclic mapping like antonymy, since applying it twice would force the offset to zero (fv-cyclic-mapping-impossibility-proof). Positively, FVs are portable across 20 in-context templates and natural-language completions (fv-portability-across-20-templates, fv-singular-plural-natural-text-portability), and they support partial vector algebra in which, for example, Last-Capital equals Last-Copy plus First-Capital minus First-Copy (fv-vector-algebra-composition). The injection formula h'_k = h_k + sum over selected heads at that layer (fv-alternative-multilayer-injection-formula) and the vocabulary-space causal-effect metric delta-logprob (fv-deltalogprob-formula) provide the operational definitions. FVs exert near-zero effect when injected at late layers, confirming they trigger intermediate feed-forward computations rather than acting on output logits (fv-late-layer-near-zero-effect), a pattern explicitly contrasted with Merullo et al.'s late-layer vector offsets (fv-merullo-contrast-late-vs-mid-layer). The research was supported by Open Philanthropy, NSF, and CAIS (fv-funding-sources), and experiments span GPT-J, GPT-NeoX, and Llama 2 at multiple scales (fv-models-tested).

The Hendel et al. thread complements this from a more task-level perspective. Their task vector theta is a single intermediate hidden state extracted at one specific layer L using a dummy query to ensure query-agnosticism (hendel-2023-task-vector-from-arrow-token, hendel-task-vector-single-intermediate-hidden-state). The optimal boundary layer L sits at a similar relative fraction of depth regardless of total model size, echoing Todd et al.'s one-third-depth finding (hendel-2023-layer-l-relative-position, fv-optimal-injection-layer-third-of-depth). Patching theta at layer L is equivalent to blocking attention to the in-context demonstrations in all layers above L, tying the vector to a specific causal pathway (hendel-2023-patching-equivalent-to-attention-masking). A striking result is that injecting a task vector for Task B into a prompt containing demonstrations for Task A causes the model to follow the vector with 77 to 95 percent accuracy, overriding the in-context examples (hendel-conflicting-task-overrides-demonstrations, hendel-conflicting-task-accuracy-range). Vocabulary projections of theta surface tokens like "English" or "participle" that never appeared verbatim in the prompt, indicating an abstracted task representation (hendel-vocabulary-projection-not-verbatim, hendel-vocabulary-projection-fr-en-tokens). t-SNE plots of 50 vectors per task show tight within-task clusters separated from cross-task clusters (hendel-t-sne-50-vectors-per-task-llama7b). The authors explicitly note they do not provide a mechanistic account of how the transformer constructs theta or uses it (hendel-no-mechanism-explanation), and scope their model to single-token-output tasks, leaving multi-step reasoning open (hendel-scope-single-token-output). Knowledge tasks draw on the counterfactual dataset from Meng et al. 2022 (hendel-knowledge-data-meng-2022), and the framework is positioned as complementary to Olsson et al.'s induction-head analysis of ICL (hendel-olsson-induction-heads-2022) and to Merullo et al.'s independent demonstration of single-vector function encoding (hendel-merullo-2023-complementary). The work is publicly available at a GitHub repository (hendel-2023-code-repository), and experiments cover LLaMA, GPT-J, and Pythia at multiple sizes (hendel-2023-models-and-tasks, hendel-gpt-j-6b-architecture, hendel-llama-7b-architecture, hendel-llama-13b-architecture, hendel-pythia-2.8b-architecture).

The vec2vec results extend the vector-as-functional-carrier idea to the problem of translating between incompatible embedding spaces. The method uses a modular architecture with space-specific input adapters, a shared MLP backbone with SiLU activations, and output adapters, trained with two-level GAN losses at both latent and output embedding spaces (vec2vec-architecture-input-output-adapters-shared-backbone, vec2vec-mlp-silu-not-cnn-no-spatial-structure, vec2vec-two-level-adversarial-training-latent-and-output). It requires zero paired examples and zero access to the source encoder (vec2vec-no-paired-data-no-encoder-access), positioning it as the first unsupervised cross-model embedding translation method grounded in a constructive version of the Platonic Representation Hypothesis (vec2vec-first-unsupervised-cross-model-embedding-translation). Performance is strong: 50K training embeddings yield cosine similarity 0.74, within 0.01 of the 1M-embedding ceiling, and full vec2vec achieves cosine 0.75 with mean rank 2.64 against a naive baseline rank of 4084 (vec2vec-data-efficiency-50k-near-1m, vec2vec-full-ablation-metrics). On the challenging Qwen-to-GTE cross-backbone pair, where Qwen is 14 times larger than the next model, vec2vec reaches mean rank 2.49 versus 425 for the best optimal-transport baseline (vec2vec-qwen-pair-outperforms-ot-rank-249-vs-425). Ablations show all three generator losses are necessary, with removing the VSP loss dropping cosine to 0.58 and rank near random (vec2vec-vsp-ablation-cos-drop, vec2vec-ablation-all-three-generator-losses-necessary). Translators generalize to unseen encoders (vec2vec-ood-robustness-unseen-encoders), though zero-shot inversion recovers document content in up to 80 percent of emails, raising privacy considerations (vec2vec-inversion-leakage-80-percent-emails-67-percent-tweets). The authors frame their results as a lower bound on inter-representation fidelity (vec2vec-lower-bound-framing), and total training compute was approximately 176 GPU days across 25 fully trained models (vec2vec-training-compute-176-gpu-days). Evaluation spans Natural Questions, TweetTopic, and MIMIC-III with four size categories and five transformer backbones (vec2vec-datasets-nq-2m-tweettopic-8192-mimic-8192).

Across all three threads, every belief in this topic is currently marked IN; no beliefs have been retracted. The beliefs mix premises (formulas, architectural specifications, dataset descriptions, funding attributions) with derived or empirical claims (impossibility proofs, performance metrics, ablation results, portability demonstrations). The unifying insight is that transformer activation-space vectors are not passive representations but active, composable, portable, and translatable carriers of executable functions, a finding that holds whether the vectors are extracted from attention-head activations (Todd et al.), from a single separator-token hidden state (Hendel et al.), or mapped across entirely different embedding spaces (Jha et al.).