lemmatization-dictionary-stemming-rules

IN premiseentries/2026/06/21/wiki-Natural_language_processing-chunk-2.md

Created 2026-06-21T09:50:10+00:00

Lemmatization uses a dictionary to map words to base forms and returns valid words; stemming uses rules to strip affixes and may produce non-words

Summary

Lemmatization looks up each word in a dictionary to find its proper base form, so the output is always a real English word. Stemming just chops off endings using mechanical rules, which can leave you with fragments that aren't actual words. This matters because any downstream task like search matching or analysis will either get clean, meaningful words or may stumble over garbage fragments depending on which approach is used.