fv-qualifying-tasks-gptj-18
IN premise — summaries/2026/08/24/todd-2023-function-vectors-s2-function-vector-causal-effects-cannot-be-recovered-from-the-.md
Created 2026-08-25T02:58:41+00:00
18 tasks qualify for GPT-J FV extraction (those where 10-shot ICL beats the majority-label baseline).
Summary
Out of the full set of tasks considered, only 18 are good enough to analyze with GPT-J's internal feature representations, because on the other tasks the model doesn't even beat the simple strategy of always guessing the most common answer. This caps the scope of the downstream functional-vector analysis and ensures we only extract patterns from tasks where the model is actually doing something meaningful.