retrieval-corpus-december-2018-wikipedia

IN premise — summaries/2026/08/24/mallen-2023-when-not-to-trust-s4-memorization-depends-on-popularity.md

Created 2026-08-25T02:58:11+00:00

The retrieval corpus used in Mallen et al. (2023) is a December 2018 Wikipedia dump.

Summary

Mallen et al. (2023) pulled their retrieval results from a snapshot of Wikipedia taken in December 2018, not a live or more recent version. This matters because it fixes the knowledge base at a specific point in time, meaning any downstream analysis of their retrieval quality is bounded by what Wikipedia articles said (and didn't yet say) as of late 2018.