hendel-conflicting-task-overrides-demonstrations

IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s7-related-work.md

Created 2026-08-25T02:58:04+00:00

When a task vector for Task B is injected into a prompt containing demonstrations for Task A, the model follows θ (Task B) with high accuracy and largely disregards the in-context demonstrations.

Summary

A task vector acts as a strong override: even when the prompt includes clear worked examples for a different task, the model will follow the vector's instruction and largely ignore those examples. This means parameter-level task specification dominates over in-context learning, so the system can treat task vectors as a reliable, high-priority control channel that supersedes whatever the few-shot demonstrations suggest.