static-vs-contextual-embedding-assignment

IN premise — summaries/2026/08/24/wiki-Word_embedding.md

Created 2026-08-24T17:11:28+00:00

Static embeddings (word2vec, GloVe, fastText) assign one fixed vector per word regardless of context, while contextual embeddings (ELMo, BERT) produce a unique vector per token occurrence, with the static approach inherently unable to distinguish polysemous senses

Summary

Static word-embedding methods like word2vec give every word one fixed numerical fingerprint, so a system built on them literally cannot tell apart multiple senses of the same word, like "bank" as a river edge versus a financial institution. Contextual models like BERT instead produce a unique fingerprint for each occurrence based on the surrounding sentence, which is the difference between a system that just looks up words and one that actually reads them in context.