kandpal-2023-oracle-retrieval-300-word-wiki

IN premise — summaries/2026/08/24/kandpal-2023-long-tail-knowledge-s4-methods-to-improve-rare-fact-learning.md

Created 2026-08-25T02:58:06+00:00

The oracle retrieval augmentation experiment provides the 300-word Wikipedia segment surrounding the ground-truth answer as context to the GPT-Neo prompt.

Summary

In this experiment, the model is handed the exact 300 words of Wikipedia text that surround the correct answer, rather than relying on a retrieval system to find them. This sets up a best-case benchmark: if the model still gets the question wrong with the perfect passage in front of it, the bottleneck is the model's reading and reasoning, not the search.