cka-sensitive-to-scale-unlike-mutual-knn

IN premise — summaries/2026-08-24/koepke-2026-back-into-cave-s2-related-work.md

Created 2026-08-24T17:11:00+00:00

Central Kernel Alignment (CKA) is sensitive to network scale and can be null-calibrated away (Gröger et al.), while the mutual kNN metric is more stable for local structural comparison, though mutual kNN was previously validated only under small-scale (≤1,024 samples) and bijective evaluation conditions (Huh et al. 2024).

Summary

Two common tools for measuring how similar two neural networks really are each have a hidden caveat: one (CKA) gives different answers depending on network size unless you apply a statistical correction, while the other (mutual kNN) is steadier for structural comparison but has only been proven reliable on small, tightly controlled test sets. In practice, neither metric can be trusted as-is when comparing larger or less-constrained models, so any downstream conclusion built on either one needs that limitation explicitly accounted for.