xie-2021-optimality-condition-signal-vs-mismatch
IN premise — summaries/2026/08/24/xie-2021-icl-bayesian-s2-in-context-learning-setting.md
Created 2026-08-25T02:58:55+00:00
Xie et al. (2021) prove that asymptotic ICL prediction error is optimal when the per-example signal about the latent concept θ* exceeds the error introduced by distribution mismatch between prompts and pretraining data.
Summary
This sets a concrete threshold for when in-context learning actually delivers its best possible performance: each example in the prompt needs to carry enough information about the underlying task to outweigh the mismatch between the prompt data and what the model saw during pretraining. Practically, it tells us why some prompts work beautifully and others fail — it is a signal-versus-noise tradeoff, and the system can diagnose poor ICL performance by checking which side of that tradeoff is losing.