prefix-tuning-vs-prompt-tuning-difference

Summary

Prefix tuning and prompt tuning are two different strategies for steering a frozen language model: prefix tuning inserts learned vectors at every layer of the transformer, giving the added parameters a say in the model's reasoning at every stage, while prompt tuning places them only at the input, so they merely set the initial context. This architectural distinction matters because it determines how deeply the small set of learned parameters can influence the model's internal computations versus just framing its starting point.