kandpal-2023-bm25-top3-wikipedia-paragraphs
IN premise — summaries/2026/08/24/kandpal-2023-long-tail-knowledge-s4-methods-to-improve-rare-fact-learning.md
Created 2026-08-25T02:58:06+00:00
The BM25 retrieval augmentation selects the top-3 scoring Wikipedia paragraphs and prepends them to both in-context training examples and the test question prompt.
Summary
Retrieved Wikipedia context is injected by pulling the three highest-scoring paragraphs via BM25 and placing them at the front of both the few-shot examples and the test question itself. This means the model's answers are directly shaped by whichever Wikipedia snippets happen to rank highest, so retrieval quality becomes a first-order bottleneck for accuracy rather than a minor detail.