gd-accuracy-agns-8-vs-512-demos
IN premise — summaries/2026/08/24/shen-2023-icl-not-gd-s6-related-work.md
Created 2026-08-25T02:58:33+00:00
Gradient descent fine-tuning with 8 demos achieves 0.42 accuracy on AGNews, rising to 0.69 with 512 demos, while ICL achieves comparable accuracy with far fewer demos.
Summary
On the AGNews text-classification task, fine-tuning a model needs a 64-fold jump in the number of example inputs to move from weak to reasonable performance, while in-context learning reaches that same level of accuracy with a fraction of the data. For the system, this means in-context learning is the more practical route when labeled examples are scarce or expensive to collect, since it avoids the steep data-hunger of gradient-based fine-tuning.