sbert-uv-product-hurts-performance
IN premise — summaries/2026/08/24/reimers-2019-sentence-bert-s6-ablation-study.md
Created 2026-08-25T02:58:29+00:00
Adding the element-wise product u*v to the siamese concatenation decreased SBERT performance (80.44 with u*v vs. 80.78 without), unlike InferSent/USE which benefit from it.
Summary
The element-wise product trick that helps InferSent and USE actually made SBERT about 0.3 points worse, meaning SBERT's transformer encoder already captures that interaction on its own and adding it as a separate feature is redundant. The practical takeaway is that architectural tricks from one sentence-embedding family don't transfer to another, and SBERT specifically should stick with the simpler concatenation-only head.