geometry-as-universal-semantic-currency-v2
IN premise
Created 2026-08-25T04:07:35+00:00
The covariance geometry serves as a unifying geometric framework for LLM semantics, underpinning what can be measured (cosine/Spearman evaluation in SBERT/MTEB), what can be modified (covariance-whitening-based editing as in ROME), what converges across architectures (cross-model universality of feature geometry), and what is structured at the interpretable level (SAE feature neighborhoods as empirical instantiations of the subordination relations in Park's orthogonality theorem)—supporting the view that it functions as a model-independent semantic structure rather than an architecture-specific artifact.
Summary
The mathematical structure governing how concepts relate inside language models is not a quirk of any particular architecture but a genuine, model-independent geometry of meaning. This matters because measurement tools, memory-editing techniques, and interpretability frameworks built on this geometry should transfer across different models, and we are discovering a real structure of semantics rather than reverse-engineering one specific design choice.
Dependents
These beliefs depend on this one:
- OUT geometry-as-universal-semantic-currency — The covariance geometry is the single operational definition of "meaning" in LLMs, simultaneously determining what can be measured (evaluation via cosine/Spearman), what can be modified (rank-one editing via C⁻¹k*), what converges across architectures (SAE/Park universality), and what is hierarchically structured (feature neighborhoods as theorem instantiations)—making it a model-independent semantic currency rather than an architecture-specific artifact.