prh-evaluation-uses-faiss-indexflatl2-and-pHash-dedup
IN premise — summaries/2026/08/24/koepke-2026-back-into-cave-sR-references-chunk-2.md
Created 2026-08-24T17:11:01+00:00
The Koepke et al. (2026) cross-modal alignment evaluation uses Faiss IndexFlatL2 for nearest-neighbor search (equivalent to cosine similarity on L2-normalized vectors), extracts DINOv2 CLS tokens from all layers (DINOv2-B = 768d, G = 1536d) and LLM hidden states from all layers with average pooling (OpenLlama-3B = 3200d), and applies pHash deduplication (64-bit fingerprint, Hamming distance ≤ 2) at large gallery scales.
Summary
This pins down the exact recipe the Koepke et al. paper uses to score cross-modal alignment: brute-force nearest-neighbor matching, multi-layer feature extraction from both the vision and language models, and a perceptual-hash filter to strip near-duplicate images from large test sets. Every downstream judgment about the quality, fairness, or comparability of those evaluation results depends on these methodological choices being treated as fixed ground truth in the system.