cgd-approximated-single-middle-layer

OUT premise — summaries/2026/08/24/shen-2023-icl-not-gd-s4-icl-demonstrations.md

Created 2026-08-25T02:58:32+00:00

In Shen et al. (2023), cGD (calibrated Gradient Descent) is approximated by performing gradient descent on a single random middle layer of LLaMA rather than fine-tuning the full model.