kalai-2023-calibration-distinct-from-kl-divergence
IN premise — summaries/2026/08/24/kalai-2023-hallucination-inevitable-sR-references.md
Created 2026-08-24T17:11:00+00:00
A model can achieve near-zero miscalibration (e.g., by outputting a random factoid) while having large KL divergence, and conversely can have small KL divergence while hallucinating — the hallucination guarantee specifically requires calibration, not just good log-likelihood or accuracy.
Summary
Getting a high accuracy score or a low divergence metric does not protect you from hallucination; a model can nail standard benchmarks and still spout confidently wrong facts, or output garbage and accidentally look "calibrated." The practical implication is that any system you build to prevent hallucination must explicitly verify calibration as its own gate, because the usual quality metrics simply do not cover that failure mode.