popqa-dataset-14k-long-tail-wikidata

IN premise — summaries/2026/08/24/mallen-2023-when-not-to-trust-s0-abstract.md

Created 2026-08-25T02:58:10+00:00

POPQA is a 14,000-question dataset constructed from Wikidata knowledge triples, with entity popularity ranked via Wikipedia page-view counts, specifically designed to probe long-tail factual knowledge.

Summary

POPQA is a 14,000-question benchmark built from Wikidata that deliberately targets obscure, rarely-looked-up facts rather than common knowledge, using Wikipedia page-view counts to rank how well-known each entity is. It matters because it sets the bar for testing whether a system can retrieve genuinely rare specific facts, not just echo what most people already know.