sbert-naive-bert-worse-than-glove

Summary

Naive BERT sentence embeddings, taken directly from the [CLS] token or a simple average of the last layer, actually lose to the much simpler approach of just averaging GloVe word vectors. This sets a low baseline: any embedding method claiming to capture sentence meaning has to beat a basic bag-of-words average, and it warns against assuming a more complex model automatically gives better sentence-level representations without deliberate training or pooling design.