geometry-ontological-status

OUT derived (depth 4)

Created 2026-08-25T03:08:54+00:00 · Reviewed 2026-08-25T04:02:18+00:00

The covariance/whitening geometry is not merely a convenient analytical tool but possesses ontological status as a genuine model-independent semantic structure, because three independent lines converge: it is the operational metric for editing and interpretation (depth-3), it is universal across architectures (depth-2), and it converges with externally-validated human-judgment metrics (depth-3).

Justifications

SL — Operational role alone could be implementation-dependent; universality alone could be a training artifact; convergence with human metrics alone could be coincidence. All three together establish the geometry as a genuine property of the model class.

Antecedents (all must be IN):

OUT covariance-geometry-as-operational-semantic-space — The covariance/whitening geometry (second-moment matrices) is the operational definition of semantic coordinate space in LLMs: it simultaneously parameterises feature interpretation (SAE decoder space, Park polytopes), similarity evaluation (cosine→Spearman pipeline), and knowledge modification (ROME rank-one updates), and this structure converges across model families.
OUT superposition-geometry-explains-universality — The cross-model universality of feature geometry (SAE features more similar across architectures than within, Park orthogonality validated on both Gemma and LLaMA) is a consequence of superposition: the over-complete compositional basis is determined by the shared semantic grammar of language, making geometric structure an architectural invariant rather than a model-specific artifact.
IN evaluation-geometry-convergence — The convergence of embedding evaluation (cosine → Spearman correlation in SBERT/MTEB) and internal geometry analysis (covariance whitening in ROME, polytope geometry in Park) on the same second-moment structure reveals that standard benchmark evaluation is measuring the same geometric quantity that governs internal feature organization—evaluation and analysis are two readouts of one space.

Dependents

These beliefs depend on this one:

OUT geometric-convergence-as-mathematical-attractor — The cross-model universality of feature geometry combined with its ontological status as a model-independent semantic object implies LLMs are converging to a shared mathematical attractor: the covariance/whitening geometry is the unique fixed point that any differentiable language model must instantiate, not an architectural artifact.
OUT geometric-framework-not-artifact — The full geometric framework of LLMs (covariance whitening, polytope decomposition, Riesz isomorphism, cross-model convergence) reflects genuine architectural structure rather than a mathematical artifact of the analysis method, provided the underlying orthogonality is not reducible to trivial set-inclusion between parent and child token sets.