raw-bert-cls-sts-correlation-29-19

Summary

BERT's default sentence representation (the CLS token) is surprisingly poor at capturing how semantically similar two sentences are, scoring roughly half as well as the much simpler GloVe word vectors. This means you cannot treat BERT's raw output as a drop-in sentence similarity signal — additional work like custom pooling or fine-tuning is needed before its representations become useful for similarity tasks.