perplexity-exponential-avg-neg-loglik

IN premise — entries/2026/06/21/wiki-Large_language_model-chunk-3.md

Created 2026-06-21T09:50:09+00:00

Perplexity is the exponential of the average negative log-likelihood per token; lower perplexity indicates better prediction of a text corpus.

Summary

Perplexity is just a single score that tells you how well a language model predicts text: the lower the number, the more confidently the model guessed each word. This matters because it gives the system one straightforward number to compare models against each other or to check whether a model has genuinely improved at a task.

Dependents

These beliefs depend on this one:

IN evaluation-landscape-spans-intrinsic-and-extrinsic-paradigms — LLM evaluation spans from intrinsic information-theoretic metrics (perplexity as the exponential of average negative log-likelihood) through multi-dimensional benchmarks (HELM evaluating accuracy, calibration, robustness, fairness, and other dimensions) to task-specific benchmarks and rankings (MMLU, Humanity's Last Exam, LMArena) — reflecting a landscape where multiple evaluation paradigms coexist, though the relationship between intrinsic quality metrics and extrinsic task performance is not explicitly characterized by these benchmarks alone.