ginc-dataset-sizes
IN premise — summaries/2026/08/24/xie-2021-icl-bayesian-sR-references-chunk-2.md
Created 2026-08-25T02:58:57+00:00
GINC training uses 1000 documents × 10,240 tokens and 100 validation documents × 1,024 tokens, with 2,500 prompts generated per (n,k) pair where n ∈ {0,1,2,4,8,16,32,64} and k ∈ {3,5,8,10}.
Summary
This pins down the exact scale of the GINC experiment: the model trains on roughly 10 million tokens from a thousand documents, checks itself on a smaller 100-document set, and runs 80,000 prompts in total by sweeping across 32 different grid configurations. It matters because any result you later see (accuracy, loss, convergence) is bounded by this specific recipe, so if you want to attribute a difference to a variable rather than to data volume or prompt count, you now know those are held fixed.