retrieval-inversion-as-geometric-signature

OUT derived (depth 6)

Created 2026-08-25T03:48:00+00:00 · Reviewed 2026-08-25T04:02:18+00:00

The inversion of accuracy trends—parametric accuracy *decreases* with document count while contextual accuracy *increases*—is the operational signature of the geometric addressability boundary: the inversion occurs precisely at the subspace boundary (where the fact direction exits the rank-one addressable space), not at a frequency threshold, making the boundary a geometric rather than statistical object.

Justifications

SL — The addressability failure (d5) and subspace boundary (d5) provide the geometric theory; the retrieval inversion (base) is the empirical signature that *confirms* the boundary is geometric. The inversion (not just the improvement) is what distinguishes a geometric boundary from a statistical one.

Antecedents (all must be IN):

OUT long-tail-as-geometric-addressability-failure — The long-tail knowledge problem (Kandpal's 10¹⁵-parameter estimate, 176B-model failure on rare facts) is fundamentally a geometric addressability failure rather than a data-scarcity or model-capacity issue: tail facts fail to acquire well-conditioned individual directions in the residual stream because their key vectors lie in the poorly-conditioned tail of the covariance spectrum, making them inaccessible to both parametric recall (no clean MLP slot) and rank-one editing (C⁻¹k* becomes ill-conditioned), and correctly routed to the contextual channel instead.
OUT parametric-write-subspace-boundary — The operational boundary between parametric recall and contextual retrieval is precisely the geometric boundary of the rank-one addressable subspace: facts whose subject-key projection aligns with the locally-stored key covariance are parametrically editable, while facts outside this subspace must be externally supplied.
IN kandpal-2023-retrieval-inverts-accuracy-trend — When retrieval-augmented context is provided, LM accuracy increases as relevant document count grows (matching human behavior), whereas closed-book accuracy decreases as rarity increases.

Dependents

These beliefs depend on this one:

OUT inversion-and-drop-as-dual-boundary — Kandpal's retrieval inversion (parametric accuracy *decreases* with document count while contextual accuracy *increases*) and ROME's multi-hop drop (40.5%→7.4% after editing) are the same geometric boundary observed from the read and write sides respectively: the rank-one addressable subspace fills the head directions parametrically but is silent on the tail, and the context window fills exactly those missing tail directions.