mteb-clustering-params
IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s8-tasks.md
Created 2026-08-25T02:58:21+00:00
MTEB clustering evaluation uses mini-batch k-means with batch_size=32 and k=number of labels, scored with V-measure
Summary
The MTEB clustering subtask is locked to a specific setup: it groups embeddings into as many clusters as there are distinct labels, runs the grouping in small chunks of 32, and grades the result with V-measure. This matters because any score you see for a model's "clustering" ability on MTEB only reflects how well it performs under those exact algorithmic and scoring choices, not under alternative clustering configurations.