mquake-dataset-3000-instances-2-4-hop

IN premise — summaries/2026/08/24/zhong-2023-mquake-sR-references.md

Created 2026-08-25T02:59:08+00:00

MQuAKE is a benchmark of 3,000 multi-hop questions (2, 3, and 4 hops) constructed from Wikidata fact chains, with variants MQuAKE-CF (counterfactual edits) and MQuAKE-T (real temporal knowledge changes).

Summary

MQuAKE is a 3,000-question test set built to check whether a knowledge system can chain together multiple facts (two, three, or four steps deep) rather than just retrieving a single lookup, and it includes extra versions that test how the system handles "what if" scenarios and real-world facts that change over time. This matters because multi-step reasoning and temporal awareness are where most fact-based systems break down, so the benchmark isolates those specific weaknesses.