cgd-single-random-middle-layer

Summary

In the Shen et al. experiments, what they call "continuous gradient descent" is actually a very narrow operation: they pick one layer from the middle of the LLaMA model at random and only adjust that layer's weights, leaving every other layer completely locked in place. This matters because any conclusions drawn from those experiments describe how a single isolated layer can shift, not how the model as a whole adapts, so the findings should not be read as evidence about full-model learning dynamics.