xie-2021-in-context-predictor-argmax

IN premise — summaries/2026/08/24/xie-2021-icl-bayesian-s1-generate-a-start-hidden-state-hstart.md

Created 2026-08-25T02:58:55+00:00

The in-context predictor is defined as fₙ(x_test) = argmax_y p(y | Sₙ, x_test), i.e., the argmax over the pretraining distribution conditioned on the full prompt, corresponding to greedy decoding in practice.

Summary

When the model predicts the next token, it simply picks the single most probable option from its pre-trained probability distribution, given everything in the prompt so far, rather than sampling or exploring alternatives. This is the standard greedy decoding step, and it means every observed "in-context learning" behavior we study is happening under this one specific, deterministic rule for choosing outputs.