hendel-2023-task-vector-activation-space
IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s1-introduction.md
Created 2026-08-25T02:58:03+00:00
The task vectors in Hendel et al. (2023) are defined in activation space (hidden representations), distinguishing them from the weight-space task vectors of Ilharco et al. (2023) Task Arithmetic.
Summary
Hendel et al. (2023) characterize a task by measuring the network's internal activations during inference, while Ilharco et al. (2023) Task Arithmetic characterize it by measuring changes in the model's learned weights. This distinction matters because the two kinds of vectors live in fundamentally different spaces, so they cannot be directly compared, added together, or swapped in for one another when the system reasons about how tasks are represented or combined.