mquake-counterfactual-validity-condition
IN premise — summaries/2026/08/24/zhong-2023-mquake-s2-problem-definition.md
Created 2026-08-25T02:59:06+00:00
A counterfactual edit in MQuAKE-CF is valid only if the updated chain yields a different answer a* ≠ a from the original answer
Summary
A counterfactual test in this question-answering system only counts if changing the input actually flips the final answer. If the system gives the same answer after the edit, the test is meaningless because it tells you nothing about which parts of the input the model actually relied on.