mteb-classification-head-params
IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s8-tasks.md
Created 2026-08-25T02:58:21+00:00
MTEB classification evaluation uses a logistic regression head with max_iter=100 trained on embeddings (no end-to-end fine-tuning)
Summary
When the MTEB benchmark scores a model on classification, it simply trains a tiny linear classifier on the fixed embedding vectors for just 100 steps, rather than updating the embedding model itself. This means the classification score is really measuring how well-separated the categories are in embedding space, not how much a model can adapt to a task, which keeps comparisons cheap and consistent across models.