task-specificity-emerges-from-readout
OUT derived (depth 2)
Created 2026-08-25T03:05:14+00:00 · Reviewed 2026-08-25T04:28:09+00:00
Task-specificity in embedding quality is a readout phenomenon: the internal feature geometry is largely model-independent (convergent across architectures), while MTEB's no-dominant-model result arises because each task's unembedding/projection head selects a different subspace of the same shared geometric structure.
Justifications
SL — Cross-model geometric convergence (Park validated on Gemma+LLaMA, SAE universal) combined with MTEB's irreducible task variance localises the source of specificity to the readout layer rather than the internal representation.
Antecedents (all must be IN):
- OUT task-specificity-vs-feature-universality — Embedding model quality is irreducibly task-specific (no single model dominates MTEB's 8 tasks) while internal feature representations are largely model-independent (SAE features transfer across architectures), separating output-space competition from internal geometric convergence.
- IN multi-model-geometric-convergence — Both the polytope/orthogonality geometry (Park, validated on Gemma-2B and LLaMA-3-8B) and sparse feature structure (SAE, universal across architectures) converge on the finding that transformer representation spaces carry model-independent geometric invariants.
Dependents
These beliefs depend on this one:
- IN evaluation-as-readout-fidelity-measure — MTEB's dataset-count-weighted bias, SBERT's task-specificity gap, and the cosine→Spearman pipeline consistency are all manifestations of a single principle: embedding benchmarks measure readout fidelity (how faithfully a linear probe extracts a specific geometric direction) rather than the completeness or quality of the underlying representation space, making leaderboard rankings a statement about calibration rather than about semantics.
- OUT mteb-aggregate-as-readout-bias — MTEB's dataset-count-weighted aggregate score measures readout calibration breadth across task types rather than internal representation quality, because task-specificity is a readout-level phenomenon operating on top of a shared geometric substrate.
- OUT mteb-no-dominant-model-as-geometric-signature — MTEB's observation that no single embedding model dominates all eight tasks is the expected geometric signature of a shared canonical semantic space probed by task-specific linear readouts, and the same second-moment structure that generates this task-specificity pattern is precisely what makes rank-one knowledge editing possible.
- OUT mteb-ranking-as-valid-comparator — The MTEB leaderboard provides a valid comparative quality signal for embedding models, with the cosine→Spearman pipeline serving as a geometry-consistent evaluation...
- OUT mteb-task-diversity-as-geometric-anisotropy — MTEB's "no dominant model" result across 8 task types is the expected geometric signature of anisotropic readout in a shared semantic space: each task type (STS, retrieval, clustering, classification) probes a different projection direction, and task-specificity is the *predicted* outcome of geometric anisotropy, not a benchmark failure or model deficiency.
- OUT mteb-task-specificity-as-shrinkage-readout — MTEB's "no dominant model" result is the readout-side signature of the same superposition that causes SAE shrinkage on the write side: the over-complete representation that prevents any finite SAE expansion ratio from fully decomposing the space also prevents any single embedding model from simultaneously optimizing all task-specific readout directions.
- OUT read-write-asymmetry — A structural asymmetry governs LLM knowledge: the read channel (feature-level interpretation via SAE, Park polytope analysis, embedding evaluation) operates in a rich, model-independent geometric space, while the write channel (editing) is impoverished to parametric rank-one MLP updates in a single layer—superposition enriches the feature space for reading but makes feature-level writing intractable.
- OUT readout-head-as-geometric-projection — The 48-point MEAN-vs-CLS pooling gap (77.03 vs 29.19) on STSb is the cleanest single experimental proof that task-specificity is a readout projection effect: the identical superposed residual stream yields radically different geometric subspaces depending solely on which linear projection (readout head) is applied.