bengio-2003-mlp-outperformed-ngram-neural-turning-point
IN premise — summaries/2026/08/24/wiki-Natural_language_processing-chunk-1.md
Created 2026-08-24T17:11:18+00:00
In 2003, Bengio et al. demonstrated that a multi-layer perceptron language model outperformed the then-best n-gram model, marking the neural turning point in NLP research
Summary
When a neural network language model first beat the best word-counting statistical model in 2003, it showed that learning rich, distributed representations of language could outperform purely frequency-based approaches. This is treated as the starting point of the neural era in NLP, meaning virtually all modern language technologies trace their design lineage back to this shift.