mteb-no-dominant-model-as-geometric-signature

OUT derived (depth 5)

Created 2026-08-25T03:15:17+00:00 · Reviewed 2026-08-25T04:02:18+00:00

MTEB's observation that no single embedding model dominates all eight tasks is the expected geometric signature of a shared canonical semantic space probed by task-specific linear readouts, and the same second-moment structure that generates this task-specificity pattern is precisely what makes rank-one knowledge editing possible.

Justifications

SL — Reframes the MTEB "no dominant model" negative observation as a positive geometric consequence: the task-specificity pattern is the readout shadow of the shared covariance geometry that also enables editing, connecting benchmark observation to the editing framework.

Antecedents (all must be IN):

OUT task-specificity-emerges-from-readout — Task-specificity in embedding quality is a readout phenomenon: the internal feature geometry is largely model-independent (convergent across architectures), while MTEB's no-dominant-model result arises because each task's unembedding/projection head selects a different subspace of the same shared geometric structure.
OUT evaluation-geometry-predicts-editability — The convergence of evaluation geometry (cosine/Spearman in SBERT/MTEB) and editing geometry (covariance whitening in ROME) on the same second-moment structure means that improving evaluation alignment and enabling reliable editing are two operational views of the same geometric optimization over the residual-stream covariance.
OUT task-specificity-vs-feature-universality — Embedding model quality is irreducibly task-specific (no single model dominates MTEB's 8 tasks) while internal feature representations are largely model-independent (SAE features transfer across architectures), separating output-space competition from internal geometric convergence.

Dependents

These beliefs depend on this one:

OUT mteb-simultaneously-measures-read-and-edit-addressability — MTEB's "no dominant model" observation is simultaneously a statement about readout diversity AND edit-addressability diversity: different embedding models optimize different directions in the same shared editable geometric space, making the benchmark a dual measure of read-quality and edit-coverage.
OUT mteb-task-specificity-as-shrinkage-readout — MTEB's "no dominant model" result is the readout-side signature of the same superposition that causes SAE shrinkage on the write side: the over-complete representation that prevents any finite SAE expansion ratio from fully decomposing the space also prevents any single embedding model from simultaneously optimizing all task-specific readout directions.