hendel-olsson-induction-heads-2022

IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s7-related-work.md

Created 2026-08-25T02:58:04+00:00

Olsson et al. (2022) identified induction heads as a primary mechanistic basis of ICL; the task-vector framework is complementary, offering a higher-level 'what is computed' perspective.

Summary

Olsson and colleagues identified a specific attention pattern (induction heads) as the low-level mechanism behind in-context learning, while the task-vector framework captures what the model is doing at a higher level of abstraction. These two views are complementary rather than competing — one explains the wiring, the other the function — so the system treats them as two layers of the same explanation of how in-context learning works.