mteb-simcse-strong-sts-weak-clustering-retrieval
IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s1-introduction.md
Created 2026-08-25T02:58:18+00:00
SimCSE performs strongly on STS tasks but poorly on clustering and retrieval, illustrating that STS performance is an unreliable proxy for general embedding quality.
Summary
Doing well on sentence-similarity benchmarks does not guarantee that the same embeddings will cluster related items or retrieve the right documents, since a model like SimCSE shows exactly that split. This means any evaluation or selection process that relies on a single task type to judge embedding quality is likely to be fooled and should test across multiple use cases before drawing conclusions.