vec2vec-full-ablation-metrics
IN premise — summaries/2026/08/24/jha-2025-vec2vec-s7-ablations.md
Created 2026-08-24T17:10:58+00:00
vec2vec (full) achieves cos 0.75, T⁻¹ 0.91, and Rank 2.64 on an 8192-record NQ evaluation against a naïve baseline of cos 0.04 and Rank 4084.15
Summary
The full vec2vec model locates the correct answer in roughly the top 3 positions out of 8,192 candidates on the Natural Questions benchmark, compared to the naïve baseline which lands it around position 4,000. This establishes that the model's embeddings are genuinely capturing meaning relevant to question answering, not just random noise, and it serves as the empirical anchor for any downstream claims about the system's performance.