popqa-popularity-measured-via-wikipedia-pageviews

IN premise — summaries/2026/08/24/mallen-2023-when-not-to-trust-s3-collect-popularity.md

Created 2026-08-25T02:58:10+00:00

Subject entity popularity in POPQA is measured using Wikipedia monthly pageviews obtained through the Wikipedia API, rather than token counts in pretraining corpora.

Summary

POPQA determines whether a topic is "well-known" by checking how many people actually look it up on Wikipedia each month, rather than by counting how often that word appears in the text it was trained on. This means the system's notion of familiarity is anchored to real human curiosity and search behavior, which can differ substantially from raw word frequency in corpora.