claude-near-perfect-kv-retrieval

IN premise — summaries/2026/08/24/liu-2023-lost-in-middle-s3-how-well-can-language-models.md

Created 2026-08-25T02:58:09+00:00

Claude-1.3 and Claude-1.3 (100K) achieved near-perfect accuracy on the synthetic key-value retrieval task across all tested context lengths (75, 140, 300 pairs), unlike most other tested models.

Summary

Claude-1.3 can reliably locate a specific piece of information buried in a long document or conversation, even as the context grows, while most other models start losing track and guessing. This matters because it suggests Claude-1.3 is a dependable choice for tasks like pulling a exact fact, setting, or reference out of a large context without hallucinating or missing it.

Dependents

These beliefs depend on this one:

OUT context-window-externalization-validation — Long-context windows (200K tokens) with near-perfect in-context key-value retrieval provide operational validation of the externalization principle at practical scale, demonstrating that in-context storage is a reliable substitute for parametric long-tail knowledge when the model can attend to all relevant information simultaneously.