hendel-task-vector-single-intermediate-hidden-state
IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s4-robustness-of-task-vectors.md
Created 2026-08-25T02:58:04+00:00
The task vector θ is a single intermediate hidden-state vector extracted at one specific layer L, not a set of learned weight updates.
Summary
The task vector being used here is one snapshot of activations pulled from a single layer of the network, not a diff of the model's learned weights. This matters because it limits what kinds of linear combinations, invariances, or arithmetic you can sensibly apply to it, and it means the vector lives in the activation space of that one layer rather than in parameter space.