kalai-hallucination-good-turing-bound

Summary

This says that when a language model hallucinates a rare one-off fact, the likelihood of doing so is roughly proportional to what fraction of its training material consists of facts it has only ever seen once. In practical terms, the model's hallucination rate on obscure facts is not some mysterious, unmeasurable quantity; it can be estimated directly from the composition of the training data using a classic statistical shortcut, giving engineers a concrete lever to reduce hallucinations by addressing how much single-occurrence content the model absorbed during training.