mteb-scope-8-task-types-58-datasets-112-languages

IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s3-the-mteb-benchmark.md

Created 2026-08-25T02:58:19+00:00

MTEB covers 8 task types, 58 datasets, 10 multilingual datasets, and 112 languages total.

Summary

MTEB is a broad benchmark, not a narrow test of a single capability. Its coverage across many task categories, dozens of datasets, and over a hundred languages means that a model's score on it reflects wide-ranging embedding quality, making it a meaningful reference point when comparing systems.