liu-2023-greedy-decoding-all-models

Summary

In the lost-in-the-middle experiment, every model was made to generate text by always picking the single most probable next word, rather than sampling or exploring alternatives. This matters because it locks the findings to one specific generation strategy, so any conclusion drawn from that study about where models "lose" information in a prompt applies only to greedy decoding and can't automatically be assumed to hold for other decoding methods like temperature sampling or beam search.