mteb-task-types-count

IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s8-tasks.md

Created 2026-08-25T02:58:20+00:00

MTEB defines 8 task types: Bitext Mining, Classification, Clustering, Pair Classification, Reranking, Retrieval, Semantic Textual Similarity, and Summarization

Summary

MTEB's evaluation framework is organized into exactly 8 distinct task categories, which sets the full scope of what the benchmark can measure. Any tool or model that integrates with MTEB needs to account for all 8 areas, and any use case that doesn't fall into one of them is simply outside the benchmark's coverage.