dai-2023-icl-seed-grid-search-seven-seeds

IN premise — summaries/2026/08/24/dai-2023-icl-gradient-descent-sA-appendix.md

Created 2026-08-24T17:10:54+00:00

Dai et al. 2023 select the ICL random seed by grid search over seeds 1 through 7 per dataset per model size, acknowledging that ICL performance is sensitive to demonstration example ordering.

Summary

Dai et al. 2023 report ICL performance that is the best result out of seven random seeds, picked after the fact for each dataset and model size. This means the published numbers reflect a lucky ordering of example demonstrations rather than a stable, representative performance, making in-context learning look more reliable and consistent than it actually is.