mteb-bitext-sts-spearman-correlation-multilingual
IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s4-additional-modalities-text-embeddings-are-chunk-2.md
Created 2026-08-25T02:58:20+00:00
In multilingual STS evaluation, MTEB reports Spearman correlation of cosine similarities as the scoring metric.
Summary
When measuring how well an embedding model captures the meaning of text pairs across multiple languages, the benchmark doesn't care about the exact similarity numbers the model outputs. It only checks whether the model's ranking of which pairs are more or less similar matches the ranking that human annotators assigned, making the score robust to outliers and scale differences.