read-write-asymmetry

OUT derived (depth 4)

Created 2026-08-25T03:07:18+00:00 · Reviewed 2026-08-25T04:02:18+00:00

A structural asymmetry governs LLM knowledge: the read channel (feature-level interpretation via SAE, Park polytope analysis, embedding evaluation) operates in a rich, model-independent geometric space, while the write channel (editing) is impoverished to parametric rank-one MLP updates in a single layer—superposition enriches the feature space for reading but makes feature-level writing intractable.

Justifications

SL — No single antecedent states the read/write gap; it emerges from combining the richness of the readout side (task-specificity is a readout phenomenon), the poverty of the write side (bounded to rank-one), and the structural cause (superposition makes features entangled and hard to individually modify).

Antecedents (all must be IN):

OUT task-specificity-emerges-from-readout — Task-specificity in embedding quality is a readout phenomenon: the internal feature geometry is largely model-independent (convergent across architectures), while MTEB's no-dominant-model result arises because each task's unembedding/projection head selects a different subspace of the same shared geometric structure.
OUT geometric-editing-addressability-bound — The covariance-geometry framework defines a precise and minimal addressable space for knowledge editing (rank-one updates to a single MLP value projection), but the combination of superposition and distributed corpus acquisition structurally bounds this to single-fact local corrections—edits cannot create novel multi-hop associations because the target knowledge was never locally consolidated in the first place.
OUT superposition-as-compositional-basis — Superposition is the fundamental compositional mechanism in LLMs: the 10–200× over-complete expansion (SAE), the key-value memory structure (ROME's W_fc/W_proj), and the direct-sum space decomposition (Park's polytope+orthogonality) are three independent geometric consequences of the same over-completeness.

Dependents

These beliefs depend on this one:

OUT read-write-asymmetry-superposition-consequence — The read-broad/write-narrow asymmetry is the direct operational consequence of over-complete superposition: reading is a simultaneous linear projection over all active features, but writing must be rank-one to avoid cross-talk in the over-complete basis, making the write channel inherently and necessarily narrower.
OUT read-write-geometric-asymmetry-topology — The LLM is architecturally a read-broad/write-narrow system: the read channel (SAE feature extraction, evaluation, retrieval) spans the full d-dimensional covariance-whitened space, while the write channel (ROME rank-one, SAE feature ablation) is restricted to a 1-dimensional key-direction subspace, making knowledge correction fundamentally more constrained than knowledge interrogation.