mquake-t-excludes-vicuna7b

IN premise — summaries/2026/08/24/zhong-2023-mquake-s5-mello-a-proposal-for-editing-large.md

Created 2026-08-25T02:59:06+00:00

MQuAKE-T excludes Vicuna-7B from evaluation due to suspected training data contamination with the temporal knowledge in the benchmark.

Summary

Vicuna-7B is deliberately left out of the MQuAKE-T benchmark because it likely saw the same time-sensitive facts during training, making any score it would get unreliable rather than a true test of knowledge. This means MQuAKE-T's results are not directly comparable to other evaluations that include Vicuna-7B, and anyone building model rankings needs to treat its absence as a data-integrity flag, not a missing data point.