evaluation-geometry-convergence
IN derived (depth 3)
Created 2026-08-25T03:07:18+00:00 · Reviewed 2026-08-25T04:28:09+00:00
The convergence of embedding evaluation (cosine → Spearman correlation in SBERT/MTEB) and internal geometry analysis (covariance whitening in ROME, polytope geometry in Park) on the same second-moment structure reveals that standard benchmark evaluation is measuring the same geometric quantity that governs internal feature organization—evaluation and analysis are two readouts of one space.
Summary
The similarity scores produced by standard embedding benchmarks and the geometric structure uncovered by internal representation analysis (covariance matrices, polytopes) are not independent measurements — they are two readouts of the same underlying shape in feature space. This means benchmark performance is not an arbitrary black-box number but a direct probe of the representation's geometry, so the task-to-task variation in scores reflects different linear angles on one shared structure rather than a flaw in the evaluation method.
Justifications
This belief has 3 justifications — it is IN if any one holds.
SL — Three independent lines (SBERT pipeline, ROME/Park geometry, the explicit geometry-probe claim) each support the convergence observation; no single one is necessary if the others hold (ANY).
Antecedents (all must be IN):
- IN embedding-evaluation-pipeline-consistency — The SBERT pipeline (NLI fine-tuning → mean pooling → cosine similarity → Spearman correlation) is the de-facto standard validated across both the original SBERT paper and the MTEB benchmark's STS evaluation protocol.
SL — Three independent lines (SBERT pipeline, ROME/Park geometry, the explicit geometry-probe claim) each support the convergence observation; no single one is necessary if the others hold (ANY).
Antecedents (all must be IN):
- IN covariance-geometry-as-canonical-tool — Independent lines of work (ROME's key-space projection and Park's unembedding whitening) converge on using empirical second-moment matrices to define the "correct" inner product for reasoning about transformer representations.
SL — Three independent lines (SBERT pipeline, ROME/Park geometry, the explicit geometry-probe claim) each support the convergence observation; no single one is necessary if the others hold (ANY).
Antecedents (all must be IN):
- IN embedding-evaluation-as-geometry-probe — The embedding evaluation pipeline (cosine similarity → Spearman correlation, validated by both SBERT and MTEB) is not an arbitrary similarity metric but a direct linear probe of the same universal feature geometry (polytopes, hierarchical orthogonality, sparse features) revealed by internal representation analysis; the observed task-specificity in MTEB scores reflects different task-specific linear projections of this shared geometric structure rather than a fundamental failure of the metric.
Dependents
These beliefs depend on this one:
- OUT evaluation-geometry-predicts-editability — The convergence of evaluation geometry (cosine/Spearman in SBERT/MTEB) and editing geometry (covariance whitening in ROME) on the same second-moment structure means that improving evaluation alignment and enabling reliable editing are two operational views of the same geometric optimization over the residual-stream covariance.
- OUT evaluation-geometry-valid-independent-of-aggregation — The convergence of the cosine→Spearman evaluation pipeline across SBERT and MTEB validates the *metric itself* as a geometric probe of semantic similarity, independent of any structural bias in how per-task scores are aggregated into a final ranking.
- OUT geometry-ontological-status — The covariance/whitening geometry is not merely a convenient analytical tool but possesses ontological status as a genuine model-independent semantic structure, because three independent lines converge: it is the operational metric for editing and interpretation (depth-3), it is universal across architectures (depth-2), and it converges with externally-validated human-judgment metrics (depth-3).