best-popqa-result-46pct
IN premise — summaries/2026/08/24/mallen-2023-when-not-to-trust-s5-non-parametric-memory-complements.md
Created 2026-08-25T02:58:11+00:00
The best reported POPQA accuracy is 46.5% achieved by GPT-3 davinci-003 with Adaptive Retrieval using GenRead and Contriever, which is 5.3% above any non-adaptive method.
Summary
Even the strongest current AI system gets most obscure factual questions wrong, capping out at roughly 46 percent accuracy on the POPQA benchmark. The fact that a dynamically adaptive retrieval strategy beats fixed-lookup approaches by more than 5 points suggests the bottleneck in rare-knowledge answering is not model size but how intelligently the system decides what to look up.