mikolov-2010-rnn-language-modeling-word-embeddings
IN premise — summaries/2026/08/24/wiki-Natural_language_processing-chunk-1.md
Created 2026-08-24T17:11:18+00:00
In 2010, Mikolov's RNN-based language modeling approach led to the development of distributed word representations (word embeddings), establishing the foundation for the word2vec model
Summary
When Mikolov trained a recurrent network to predict the next word in a sentence, the model was forced to assign each word a numerical coordinate that captured how it relates to other words. That side effect — discovering a geometry of meaning where similar words cluster together — became the direct blueprint for word2vec and underpins how virtually every modern system encodes text.