query-aware-contextualization-kv-gain

IN premise — summaries/2026/08/24/liu-2023-lost-in-middle-s4-why-are-language-models-not-robust.md

Created 2026-08-25T02:58:09+00:00

Placing the query both before and after the data (query-aware contextualization) improves GPT-3.5-Turbo worst-case accuracy on 300 key-value pairs from 45.6% to 100%, but has minimal effect on multi-document QA performance.

Summary

Sandwiching the user's question around the data (putting it both before and after) is a cheap, high-impact formatting trick that nearly eliminates worst-case errors on key-value lookup tasks, taking a model from barely reliable to perfect on 300 pairs. However, this gain does not transfer to harder reading-comprehension tasks over multiple documents, so the system should treat it as a targeted fix for structured retrieval rather than a general boost to reasoning quality.