hendel-2023-patching-equivalent-to-attention-masking

IN premise — summaries/2026/08/24/hendel-2023-icl-task-vectors-s3-validity-of-the-hypothesis-class-view.md

Created 2026-08-25T02:58:03+00:00

Patching θ at layer L during the f forward pass in Hendel et al. (2023) is equivalent (ignoring positional embeddings) to blocking attention to the demonstrations S in all layers above L.

Summary

In the Hendel et al. (2023) setup, swapping out a layer's hidden-state values is mathematically the same as telling every later layer to stop looking at the prompt's example inputs. This means activation-patching experiments can be read as a clean probe of how much the model relies on those demonstrations beyond a given depth, giving a single unified interpretation for two techniques that otherwise look unrelated.