two-stage-nli-sts-training-improvement

IN premise — summaries/2026/08/24/reimers-2019-sentence-bert-s4-evaluation-semantic-textual.md

Created 2026-08-25T02:58:28+00:00

Two-stage training (NLI then STS) yields a 1–2 point Spearman improvement for SBERT and 3–4 points for the BERT cross-encoder over STS-only training.

Summary

Training a sentence-embedding model first on inference tasks (recognizing whether one sentence logically follows from another) and then on similarity ranking gives it a measurable edge over training on similarity alone. This matters because it shows that the order and variety of training signals shape how well the model ultimately distinguishes near-duplicate meanings from unrelated ones, with cross-encoder architectures benefiting roughly double as much as bi-encoder ones from that extra first stage.