hendel-task-arithmetic-ilharco-2023
IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s8-conclusions.md
Created 2026-08-25T02:58:04+00:00
Ilharco et al. (2023) work with weight-space vectors for task manipulation via task arithmetic; Hendel et al. show that in-context demonstrations produce a similar functional vector in activation space.
Summary
Two different ways of steering a model do the same thing underneath: nudging its weights directly (task arithmetic) and simply showing it a few examples in its input (in-context learning) both produce an equivalent functional signal inside the network. This matters because it means in-context learning is not some mysterious emergent trick; it is functionally the same kind of small directional shift that weight-space manipulation produces, so tools and intuitions built for one apply to the other.