post-calibration-cka-correlation-drops-while-mknn-persists

IN premise — summaries/2026/08/24/aristotelian-2026-s6-experiments.md

Created 2026-08-24T17:10:50+00:00

After null-calibration, the Pearson correlation with model capability drops for global spectral metrics (linear CKA: 0.86→0.45; Procrustes: 0.89→0.39) while local neighborhood metrics remain essentially unchanged (mKNN: ~0.85; CKNNA: ~0.87), indicating global spectral convergence was largely an artifact of width and depth confounders.

Summary

Once you strip out the fact that bigger and deeper models naturally score higher on broad structural measures, those global metrics barely track real capability anymore. The takeaway is that the local neighborhood measures (mKNN, CKNNA) are the trustworthy signals of representational quality, because they hold up regardless of model size, while the global spectral ones were mostly just reflecting "bigger model" all along.