readout-head-as-geometric-projection
OUT derived (depth 3)
Created 2026-08-25T04:19:07+00:00 · Reviewed 2026-08-25T04:28:09+00:00
The 48-point MEAN-vs-CLS pooling gap (77.03 vs 29.19) on STSb is the cleanest single experimental proof that task-specificity is a readout projection effect: the identical superposed residual stream yields radically different geometric subspaces depending solely on which linear projection (readout head) is applied.
Justifications
SL — The pooling ablation is the most controlled empirical demonstration of the readout-diversity hypothesis; both the data point and the theoretical framework are required to make the connection.
Antecedents (all must be IN):
- IN sbert-mean-vs-cls-stsb-77-03-vs-29-19 — On STSb, MEAN pooling achieves 77.03 average Spearman correlation versus 29.19 for the CLS-token vector in raw BERT.
- OUT task-specificity-emerges-from-readout — Task-specificity in embedding quality is a readout phenomenon: the internal feature geometry is largely model-independent (convergent across architectures), while MTEB's no-dominant-model result arises because each task's unembedding/projection head selects a different subspace of the same shared geometric structure.