hendel-2023-core-decomposition-equation
IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s2-a-hypothesis-class-view-of-icl.md
Created 2026-08-25T02:58:03+00:00
Hendel et al. (EMNLP 2023) formalize ICL as T([S, x]) = f(x; A(S)), where A maps demonstrations S to a task vector θ independently of query x, and f applies θ to x to produce output.
Summary
Hendel and colleagues show that in-context learning splits cleanly into two steps: the model first distills the example prompts into a fixed task definition, then applies that definition to whatever new input arrives. This matters because it means the examples and the query don't interact in some tangled way; the examples essentially write a set of instructions, and the model just executes them, so improving performance comes down to making that implicit instruction-extraction step more faithful.