cgd-learning-rates-tested

IN premise — summaries/2026/08/24/shen-2023-icl-not-gd-s1-icl-demonstrations.md

Created 2026-08-25T02:58:31+00:00

The cGD experiments in Shen et al. test four learning rates: 1e-4, 5e-4, 1e-3, and 5e-3.

Summary

The cGD results from Shen et al. only cover a narrow band of learning rates, from 0.0001 up to 0.005, so any claim about how cGD behaves at faster or slower rates goes beyond what that paper actually measured. This sets a hard boundary on what conclusions the system can safely draw from those experiments.