rlhf-reduces-positional-bias-marginally

IN premise — summaries/2026/08/24/liu-2023-lost-in-middle-s20-t-otal-retrieved-documents.md

Created 2026-08-25T02:58:08+00:00

RLHF fine-tuning reduces the best-to-worst positional accuracy gap in Llama-2 13B from 20 points (base) to approximately 10 points (chat), but does not eliminate the U-shaped curve.

Summary

RLHF fine-tuning does shrink the model's positional bias, cutting the performance gap between its best and worst positions roughly in half, but it does not remove the underlying tendency to favor the first and last items in a list while performing worst in the middle. In practice, this means that even a chat-tuned model still gives unfair advantage to whichever options happen to sit at the edges of a prompt, so any ranking or multi-item evaluation built on it remains systematically skewed unless the positions are shuffled.