mteb-task-diversity-as-geometric-anisotropy

OUT derived (depth 9)

Created 2026-08-25T03:48:00+00:00 · Reviewed 2026-08-25T04:02:18+00:00

MTEB's "no dominant model" result across 8 task types is the expected geometric signature of anisotropic readout in a shared semantic space: each task type (STS, retrieval, clustering, classification) probes a different projection direction, and task-specificity is the *predicted* outcome of geometric anisotropy, not a benchmark failure or model deficiency.

Justifications

SL — The readout-fidelity principle (d3) frames evaluation as geometric projection; the MTEB read+edit signature (d7) shows tasks probe both channels; task-specificity-as-readout (d2) establishes the anisotropy mechanism. Together they reframe "no dominant model" as a *prediction* of the geometry.

Antecedents (all must be IN):

  • IN evaluation-as-readout-fidelity-measure — MTEB's dataset-count-weighted bias, SBERT's task-specificity gap, and the cosine→Spearman pipeline consistency are all manifestations of a single principle: embedding benchmarks measure readout fidelity (how faithfully a linear probe extracts a specific geometric direction) rather than the completeness or quality of the underlying representation space, making leaderboard rankings a statement about calibration rather than about semantics.
  • OUT mteb-simultaneously-measures-read-and-edit-addressability — MTEB's "no dominant model" observation is simultaneously a statement about readout diversity AND edit-addressability diversity: different embedding models optimize different directions in the same shared editable geometric space, making the benchmark a dual measure of read-quality and edit-coverage.
  • OUT task-specificity-emerges-from-readout — Task-specificity in embedding quality is a readout phenomenon: the internal feature geometry is largely model-independent (convergent across architectures), while MTEB's no-dominant-model result arises because each task's unembedding/projection head selects a different subspace of the same shared geometric structure.