icl-logit-via-dot-product-last-hidden-state-answer-embedding
IN premise — summaries/2026/08/24/dai-2023-icl-gradient-descent-s1-introduction.md
Created 2026-08-24T17:10:53+00:00
In the ICL classification formulation, the logit for answer y_j is computed as the dot product of the last-token hidden state M(I) with the output word embedding e_{y_j}, followed by softmax, with the predicted label selected via argmax over the restricted candidate set Y.
Summary
In-context learning for classification works by treating label selection as a next-word prediction problem: the model's final internal representation is compared (via a dot product) against the vocabulary embedding of each allowed label, and whichever label "looks most like the next word" wins. This means the system doesn't use a separate classifier head; it just constrains the normal text-generation output to the set of valid answers, so everything hinges on the last hidden state capturing enough of the task to disambiguate among the candidates.