mteb-pair-class-main-metric
IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s8-tasks.md
Created 2026-08-25T02:58:22+00:00
The main metric for MTEB Pair Classification is average precision (AP) computed via cosine similarity
Summary
When a text embedding model is scored on the Pair Classification task, its performance is measured by average precision calculated from cosine similarity between embeddings, not by a simple accuracy or F1 score. This sets the standard for what "good" looks like in that task and means any comparison or ranking of models on it should use that specific number as the yardstick.