llm-u-shaped-performance-curve-universal
IN premise — summaries/2026/08/24/liu-2023-lost-in-middle-s20-t-otal-retrieved-documents.md
Created 2026-08-25T02:58:08+00:00
LLMs exhibit a U-shaped performance curve in multi-document QA where accuracy is highest when the relevant document is at the beginning or end of context and lowest at the middle, across all tested models including GPT-4, Claude-1.3, GPT-3.5-Turbo, MPT-30B, LongChat-13B, and Llama-2.
Summary
When you feed a large language model a long batch of documents and ask it a question, it reliably does a worse job when the answer is buried in the middle of that batch compared to when it's at the very start or end. This isn't a bug in one specific model — it shows up across every major LLM tested, which means it's a fundamental architectural limitation, not something a future patch will fix. In practice, any system that relies on LLMs to pull facts from long contexts has to treat document order as a critical design parameter and actively work around the "lost in the middle" effect through chunking, reordering, or retrieval strategies.