confidence-measurement-three-token-normalization
IN premise — summaries/2026/08/24/xie-2024-chameleon-sloth-sA-appendix-chunk-1.md
Created 2026-08-25T02:59:00+00:00
Confidence is measured by normalizing log probabilities over three candidate answer tokens (memory answer, counter-answer, uncertain) and reporting the distribution of the first generated token's probability.
Summary
The system gauges how sure it is by splitting probability across exactly three outcomes — the stored answer, a contradicting answer, and genuine uncertainty — rather than giving a single yes-or-no confidence score. This means every confidence reading is really a three-way tradeoff, and the system can never express certainty outside those three categories, which keeps its self-assessment structured and comparable across questions.