mallen-2023-first-query-level-retrieval

IN premise — summaries/2026/08/24/mallen-2023-when-not-to-trust-s3-collect-popularity.md

Created 2026-08-25T02:58:10+00:00

Mallen et al. (2023) are the first to study per-query (rather than per-token) retrieval decisions in the context of retrieval-augmented LM prompting without fine-tuning.

Summary

Mallen and colleagues (2023) identified a practical gap: prior work on retrieval-augmented language models decided whether to pull in external information token by token, but no one had yet asked whether you can make that call once per full query instead, without having to fine-tune the model. Recognizing this as a distinct, previously unexamined design choice gives the system a clearer vocabulary for comparing retrieval strategies and for deciding which level of granularity is appropriate for a given task.