icl-gd-order-sensitivity-discriminator
IN premise — summaries/2026/08/24/shen-2023-icl-not-gd-s0-abstract.md
Created 2026-08-25T02:58:31+00:00
ICL and GD exhibit different sensitivity to the order in which demonstrations are presented, which is evidence against their functional equivalence.
Summary
When the order of example inputs is shuffled, In-Context Learning and Gradient Descent respond in measurably different ways. That asymmetry suggests they are not running the same underlying process, even when they occasionally produce similar outputs, which matters because it limits how freely the system can swap one for the other.