mteb-clustering-vmeasure-kmeans-batch32
IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s3-the-mteb-benchmark.md
Created 2026-08-25T02:58:19+00:00
MTEB clustering evaluation uses mini-batch k-means with batch size 32 and V-measure as the primary metric.
Summary
The MTEB benchmark scores clustering quality by running a lightweight, batch-oriented version of k-means (processing 32 points at a time) and then measuring how well the resulting groups match the true labels using a V-measure score. This matters because any interpretation of a model's "clustering score" on MTEB is only as meaningful as these specific methodological choices, and switching the algorithm or metric would produce a different ranking of models.