prefix-tuning-continuous-vector-per-layer

Summary

Instead of retraining a Transformer's billions of parameters to adapt it to a new task, this approach inserts a small, learnable "hint" vector at the start of each layer and trains only that vector while leaving the original model completely untouched. The practical implication is that task adaptation becomes dramatically cheaper in both compute and memory, since the set of trainable parameters shrinks by orders of magnitude.