hendel-2023-task-vector-from-arrow-token
IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s2-a-hypothesis-class-view-of-icl.md
Created 2026-08-25T02:58:03+00:00
In Hendel et al. (2023), the task vector θ is extracted as the hidden representation of the '→' (separator) token at an intermediate layer L, computed using a dummy query (e.g., 'Plum') to ensure query-agnosticism.
Summary
Hendel and colleagues extract a model's internal "task identity" by reading the hidden state of a simple arrow token at a middle layer, deliberately using a filler word so the result doesn't accidentally depend on the specific question being asked. This matters because it gives the system a stable, content-free fingerprint of what a task *is*, separate from any particular example, which makes the representation reusable across different queries.