hendel-2023-a-f-separation-accuracy

IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s3-validity-of-the-hypothesis-class-view.md

Created 2026-08-25T02:58:03+00:00

The A/f two-stage decomposition in Hendel et al. (2023) preserves 80–90% of standard ICL accuracy, while the no-demonstration baseline T([x, →]) achieves only 10–20%.

Summary

Hendel et al. (2023) showed that in-context learning works by first attending to the example prompts and then applying a learned transformation to the new input, and that this two-step structure is doing nearly all the heavy lifting: keep it and you retain most of the accuracy, but strip out the examples and the model drops to near-chance performance. This means the model is not vaguely "generalizing" from a prompt, but following a specific, decomposable pipeline, which gives us a clean place to intervene, audit, or replace individual stages.