context-window-ephemeral-edit

OUT derived (depth 4)

Created 2026-08-25T03:08:54+00:00 · Reviewed 2026-08-25T04:02:18+00:00

The long-context window with near-perfect in-context key-value retrieval functions as an ephemeral, weight-free knowledge editing mechanism: injecting a fact into the 200K-token context is functionally equivalent to a rank-one edit that bypasses the parametric write subspace, providing a complementary write channel with zero persistence cost.

Justifications

SL — The duality establishes two channels, the context window validates the external channel's fidelity, and the editing bound shows why the parametric channel is constrained—only together do they imply the context window is the "unconstrained write" complement.

Antecedents (all must be IN):

OUT parametric-contextual-knowledge-duality — LLMs maintain two functionally independent knowledge channels: parametric (editable via ROME's rank-one weight update) and contextual (supplied via retrieved passages that invert accuracy trends), which operate on separate mechanisms and can be independently manipulated.
OUT context-window-externalization-validation — Long-context windows (200K tokens) with near-perfect in-context key-value retrieval provide operational validation of the externalization principle at practical scale, demonstrating that in-context storage is a reliable substitute for parametric long-tail knowledge when the model can attend to all relevant information simultaneously.
OUT geometric-editing-addressability-bound — The covariance-geometry framework defines a precise and minimal addressable space for knowledge editing (rank-one updates to a single MLP value projection), but the combination of superposition and distributed corpus acquisition structurally bounds this to single-fact local corrections—edits cannot create novel multi-hop associations because the target knowledge was never locally consolidated in the first place.

Dependents

These beliefs depend on this one:

OUT context-write-unbounded-complement — The context window functions as the unbounded write channel that complements the rank-one parametric write: where parametric editing is confined to a single MLP's addressable subspace, the context channel injects arbitrary quantities of verified knowledge, making the two write channels capacity-complementary rather than redundant.
OUT scoped-factual-knowledge-lifecycle — For single-hop factual (entity-relation-object) knowledge specifically, the full lifecycle—distributed corpus acquisition → local MLP key-value storage → parametric recall → contextual supplementation → rank-one correction—forms a coherent, validated closed loop in which each stage is geometrically well-defined and operationally distinct.