sbert-mean-vs-cls-stsb-77-03-vs-29-19
IN premise — summaries/2026/08/24/reimers-2019-sentence-bert-s4-evaluation-semantic-textual.md
Created 2026-08-25T02:58:28+00:00
On STSb, MEAN pooling achieves 77.03 average Spearman correlation versus 29.19 for the CLS-token vector in raw BERT.
Summary
In raw, unfine-tuned BERT, averaging all the word vectors together is a far more effective way to capture a sentence's overall meaning than simply reading the special [CLS] token, as measured by how well the model ranks sentence-pair similarity. This matters because it tells a system that without further training, the [CLS] token is essentially unreliable as a sentence representation, and any downstream task relying on it will suffer.
Dependents
These beliefs depend on this one:
- OUT readout-head-as-geometric-projection — The 48-point MEAN-vs-CLS pooling gap (77.03 vs 29.19) on STSb is the cleanest single experimental proof that task-specificity is a readout projection effect: the identical superposed residual stream yields radically different geometric subspaces depending solely on which linear projection (readout head) is applied.