sbert-inference-cosine-similarity-o-n

IN premise — summaries/2026/08/24/reimers-2019-sentence-bert-s3-model.md

Created 2026-08-25T02:58:28+00:00

At inference, SBERT computes similarity via cosine-similarity between two fixed-size embeddings, which is O(n) per pair, unlike poly-encoders requiring O(n²) attention-based scoring.

Summary

At inference time, SBERT just compares two pre-computed fixed-length vectors with a simple dot-product operation, so scoring a pair of sentences is fast and linear in the vector size. This is a practical speed advantage over poly-encoder architectures, which must recompute attention over all token pairs and scale quadratically, making SBERT much cheaper to run at scale.