parametric-contextual-knowledge-duality
OUT derived (depth 1)
Created 2026-08-25T03:00:40+00:00 · Reviewed 2026-08-25T04:28:09+00:00
LLMs maintain two functionally independent knowledge channels: parametric (editable via ROME's rank-one weight update) and contextual (supplied via retrieved passages that invert accuracy trends), which operate on separate mechanisms and can be independently manipulated.
Justifications
This belief has 2 justifications — it is IN if any one holds.
SL — ROME achieves 99–100% single-fact efficacy through weight modification; Kandpal shows providing the same fact in context flips accuracy from below to above human level. These are orthogonal knowledge-access axes that can be independently targeted.
Antecedents (all must be IN):
- IN rome-quantitative-benchmark-results — On COUNTERFACT, ROME achieves ~99–100% efficacy and ~75–79% neighborhood specificity, while fine-tuning achieves 100% efficacy but only ~10–40% specificity.
Unless (any of these IN defeats this justification):
- IN parametric-contextual-knowledge-duality-v2 — Evidence suggests LLMs exhibit at least two distinguishable knowledge pathways: a parametric channel amenable to targeted editing via ROME's rank-one weight update (achieving ~99–100% efficacy with ~75–79% neighborhood specificity, in contrast to fine-tuning's 100% efficacy but only ~10–40% specificity), and a contextual channel in which retrieved passages alter the accuracy–rarity relationship (accuracy increases with relevant document count, inverting the closed-book trend). These pathways display distinct behavioral profiles and appear to be amenable to different manipulation strategies, though the available evidence does not establish full functional independence or confirm that they rely on entirely separate internal mechanisms.
SL — ROME achieves 99–100% single-fact efficacy through weight modification; Kandpal shows providing the same fact in context flips accuracy from below to above human level. These are orthogonal knowledge-access axes that can be independently targeted.
Antecedents (all must be IN):
- IN kandpal-2023-retrieval-inverts-accuracy-trend — When retrieval-augmented context is provided, LM accuracy increases as relevant document count grows (matching human behavior), whereas closed-book accuracy decreases as rarity increases.
Unless (any of these IN defeats this justification):
- IN parametric-contextual-knowledge-duality-v2 — Evidence suggests LLMs exhibit at least two distinguishable knowledge pathways: a parametric channel amenable to targeted editing via ROME's rank-one weight update (achieving ~99–100% efficacy with ~75–79% neighborhood specificity, in contrast to fine-tuning's 100% efficacy but only ~10–40% specificity), and a contextual channel in which retrieved passages alter the accuracy–rarity relationship (accuracy increases with relevant document count, inverting the closed-book trend). These pathways display distinct behavioral profiles and appear to be amenable to different manipulation strategies, though the available evidence does not establish full functional independence or confirm that they rely on entirely separate internal mechanisms.
Dependents
These beliefs depend on this one:
- OUT context-window-ephemeral-edit — The long-context window with near-perfect in-context key-value retrieval functions as an ephemeral, weight-free knowledge editing mechanism: injecting a fact into the 200K-token context is functionally equivalent to a rank-one edit that bypasses the parametric write subspace, providing a complementary write channel with zero persistence cost.
- OUT distributed-acquisition-bounds-editing-scope — Knowledge editing is fundamentally a correction mechanism (fixing mis-encoded local associations) rather than a creation mechanism, because novel factual knowledge requires the distributed corpus exposure that a single rank-one update on one MLP layer cannot replicate.
- OUT knowledge-lifecycle-closed-loop — The full knowledge lifecycle in LLMs—distributed corpus acquisition → local MLP key-value storage → parametric recall → rank-one editing correction—forms a closed loop unified by the covariance geometry, where each stage is addressable in the same second-moment coordinate space.
- OUT knowledge-routing-architecture — The LLM implements a natural knowledge-routing architecture: well-supported facts are parametrically stored in local MLP key-value memory (head of the distribution), while rare/long-tail facts are architecturally routed through the contextual channel (retrieval, extended context windows), with the routing boundary determined by corpus document frequency rather than an arbitrary design choice.
- OUT parametric-sufficiency-for-head-facts — Parametric MLP key-value storage is a sufficient mechanism for well-supported (head-of-distribution) knowledge, with the parametric/contextual duality providing a clean division of labor for knowledge storage and retrieval...
- OUT riesz-map-as-key-value-semantic-bridge — The Riesz isomorphism under the causal inner product is the mathematical "compiler" that translates between ROME's key representation (MLP input activation) and value representation (MLP output projection), unifying the parametric knowledge storage mechanism as a single dual-geometric object rather than two independent weight matrices.