reader-saturation-20-documents
IN premise — summaries/2026/08/24/liu-2023-lost-in-middle-s6-related-work.md
Created 2026-08-25T02:58:09+00:00
In open-domain QA, language-model reader accuracy plateaus at approximately 20 retrieved documents; beyond that, additional documents yield only ~1.5% (GPT-3.5-Turbo) or ~1% (Claude-1.3) marginal accuracy gains while retriever recall continues to improve.
Summary
A language model can only effectively use about 20 retrieved documents before extra ones add almost nothing to answer quality, even though the retrieval pipeline can keep finding more relevant material. This means the system has a hard practical ceiling on context it can productively consume, so investing in retrieving beyond that threshold wastes compute without meaningfully improving answers.