gsm8k-reverses-difficulty-inversion
IN premise — summaries/2026/08/24/convergence-without-understanding-2026-s3-results.md
Created 2026-08-24T17:10:52+00:00
GSM8K (mathematics) reverses the difficulty inversion pattern: hard-problem CKA = 0.76 and easy-problem CKA = 0.89 (gap −0.122), unlike science, truthfulness, and commonsense domains which show the standard inversion, attributed to algorithmic compressibility constraining valid solution strategies.
Summary
Math word problems (GSM8K) behave differently from science, truthfulness, and commonsense tasks: the model actually performs better on the easier problems than the harder ones, which is the opposite of the pattern seen everywhere else. This reversal happens because math solutions tend to follow a narrow set of compressible step-by-step procedures, so there are fewer alternative paths to a correct answer, and this matters for how we interpret model capability across domains rather than assuming one performance profile applies to all reasoning types.