parametric-contextual-knowledge-duality-v2
IN premise
Created 2026-08-25T04:31:20+00:00
Evidence suggests LLMs exhibit at least two distinguishable knowledge pathways: a parametric channel amenable to targeted editing via ROME's rank-one weight update (achieving ~99–100% efficacy with ~75–79% neighborhood specificity, in contrast to fine-tuning's 100% efficacy but only ~10–40% specificity), and a contextual channel in which retrieved passages alter the accuracy–rarity relationship (accuracy increases with relevant document count, inverting the closed-book trend). These pathways display distinct behavioral profiles and appear to be amenable to different manipulation strategies, though the available evidence does not establish full functional independence or confirm that they rely on entirely separate internal mechanisms.
Summary
Large language models appear to draw on knowledge through at least two somewhat separate channels: one baked into their internal weights that can be surgically edited with high precision and minimal collateral change, and another shaped by retrieved context that shifts the model's accuracy-rarity behavior in ways the closed-book version does not. This matters because it implies you can target and modify specific facts without broadly destabilizing the model, and that adding relevant documents isn't mere reinforcement but changes the model's behavioral pattern in a distinct way, though the two channels may not be as cleanly separated as the current evidence suggests.
Dependents
These beliefs depend on this one:
- OUT parametric-contextual-knowledge-duality — LLMs maintain two functionally independent knowledge channels: parametric (editable via ROME's rank-one weight update) and contextual (supplied via retrieved passages that invert accuracy trends), which operate on separate mechanisms and can be independently manipulated.