hendel-2023-models-and-tasks
IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s3-validity-of-the-hypothesis-class-view.md
Created 2026-08-25T02:58:03+00:00
Hendel et al. (2023) validate the task-vector decomposition across 18 tasks in 4 categories (algorithmic, translation, linguistic, factual knowledge) using LLaMA 7B/13B/30B, GPT-J 6B, and Pythia 2.8B/6.9B/12B.
Summary
The idea that a model's capability for a specific task can be isolated as a separable "vector" in its weights holds up broadly — it isn't limited to one model family, one size, or one type of task. This matters because it suggests model capabilities are modular to some degree, opening the door to combining, removing, or analyzing individual skills as distinct components rather than treating the model as an opaque black box.