icl-zero-shot-exceeds-few-shot-low-entropy
IN premise — summaries/2026/08/24/xie-2021-icl-bayesian-s4-simulations.md
Created 2026-08-25T02:58:56+00:00
Under low-entropy transition matrices (temperature 0.01, 12 concepts, vocabulary 100), zero-shot accuracy initially exceeds few-shot accuracy on GINC before recovering with more examples, attributed to the few-shot prompt format acting as a distractor.
Summary
When the underlying task is highly predictable, giving the model examples in a structured few-shot prompt can actually hurt its performance at first, because the prompt format itself becomes a distraction rather than a helpful cue. This suggests that in stable, low-uncertainty environments, a simpler zero-shot approach may outperform example-based prompting until enough examples are provided to overcome the format's interference.