mteb-multilingual-dataset-language-counts
IN premise — summaries/2026/08/24/muennighoff-2022-mteb-sR-references.md
Created 2026-08-25T02:58:23+00:00
MTEB multilingual dataset language counts are: Massive=51, Tatoeba=112, STS22=18, STS17=11, AmazonReviews=6, AmazonCounterfactual=4.
Summary
This records the actual number of languages each dataset in the MTEB multilingual benchmark covers, ranging from 4 (AmazonCounterfactual) up to 112 (Tatoeba). It matters because it shows that language coverage varies wildly across tasks, so a model that scores well on the broad datasets like Tatoeba or Massive is being tested very differently from one scored on the narrower sets, and any aggregate "multilingual" score should be interpreted with that spread in mind.