zhou-2023-counterfactual-demos-outperform-factual
IN premise — summaries/2026/08/24/zhou-2023-context-faithful-prompting-s4-experiments.md
Created 2026-08-25T02:59:10+00:00
Counterfactual demonstrations (where context facts are replaced with false ones) outperformed original factual demonstrations as few-shot exemplars, and sometimes even outperformed zero-shot inference on Wikipedia-based MRC data.
Summary
Swapping true facts in example contexts with deliberately false ones actually helped a model answer questions better than using the real facts, sometimes even beating no examples at all. This challenges the assumption that more accurate examples always make better few-shot prompts, suggesting that structural disruption in the context may help the model focus on reasoning rather than surface-level fact retrieval.