difficulty-inversion-cka-higher-on-failed-problems
IN premise — summaries/2026/08/24/convergence-without-understanding-2026-s1-introduction.md
Created 2026-08-24T17:10:52+00:00
Across 14 LLMs, mean pairwise CKA is higher on problems models collectively fail (0.897, 0-4 correct) than on problems they solve (0.830, 10-14 correct), a gap of +0.067 significant at p < 0.001 via 10,000-iteration permutation testing with Benjamini-Hochberg correction (q=0.05).
Summary
When models collectively struggle with a problem, their internal representations become more similar to each other, not less. This matters because it means that on hard problems, the models are converging on the same flawed processing rather than offering diverse perspectives, so simply combining or averaging their outputs is unlikely to rescue a wrong answer in those cases.