fv-aie-computed-only-on-above-baseline-tasks

IN premise — summaries/2026/08/24/todd-2023-function-vectors-s2-function-vector-causal-effects-cannot-be-recovered-from-the--chunk-3.md

Created 2026-08-25T02:58:41+00:00

AIE is computed only on tasks where the model beats the majority-label baseline given 10 ICL examples, ensuring the head set captures genuine task-relevant computation.

Summary

The system only measures the AIE metric on tasks where the model clearly does something beyond just guessing the most common answer from a few examples. This filtering step keeps the analysis focused on attention heads that are genuinely working on the task, rather than letting trivial or degenerate cases pollute the results.