llama2-70b-strategyqa-flip-direction-100pct-true-to-false
IN premise — summaries/2026/08/24/xie-2024-chameleon-sloth-sA-appendix-chunk-1.md
Created 2026-08-25T02:58:59+00:00
On StrategyQA, 100% of observed answer flips for Llama2-70B are in the True→False direction.
Summary
When Llama2-70B gives inconsistent answers on multi-step reasoning questions, the drift is always in one direction: it moves from saying "yes" to saying "no," and never the reverse. This means the model treats positive claims as provisional and is more willing to retract them, so any system relying on it should expect "yes" answers to be less stable than "no" answers under repeated sampling.