mteb-reranking-main-metric

IN premise — summaries/2026/08/24/muennighoff-2022-mteb-s8-tasks.md

Created 2026-08-25T02:58:20+00:00

The main metric for MTEB reranking evaluation is MAP (mean average precision)

Summary

When the MTEB benchmark reports reranking results, the number you are looking at is mean average precision, which rewards a model for placing the most relevant documents near the top of the ranked list. This matters because every performance comparison, model selection decision, or improvement target in the system should be read through that single lens: higher MAP means the reranker is better at surfacing the right items first, which is what downstream search and retrieval tasks actually depend on.