convergence-is-necessary-not-contingent-v2
IN premise
Created 2026-08-25T03:27:03+00:00
Cross-model geometric convergence (SAE feature similarity, Park orthogonality) is strongly predicted by the superposition framework: over-complete representation (m > d) in a shared substrate motivates a common covariance structure, providing a unified explanation for why geometric invariants appear across architectures. Empirical convergence (validated on Gemma-2B, LLaMA-3-8B, with SAE features 'mostly similar' across models) is consistent with this theoretical account and argues against a pure contingent coincidence, though the evidence establishes a well-motivated expectation rather than a proven logical necessity for all systems that encode concepts in an over-complete basis.
Summary
Different AI models tend to organize their internal concepts in surprisingly similar geometric ways, and this is best explained by a shared mathematical constraint: when a model must pack more ideas into fewer dimensions than are available, the geometry of that compression pushes any such system toward the same structural shape, rather than the similarity being a lucky accident. That matters because it gives us a principled reason to expect cross-model comparisons, shared interpretability tools, and geometric invariants to keep holding as new architectures come out, even though it stops short of guaranteeing the pattern for every possible system.
Dependents
These beliefs depend on this one:
- OUT convergence-is-necessary-not-contingent — Cross-model geometric convergence (SAE feature similarity, Park orthogonality) is a logical necessity of superposition in a shared residual stream rather than a contingent empirical coincidence: any system that encodes d concepts in an over-complete m > d basis within a common substrate MUST produce the same covariance structure