llama2-7b-confidence-over-95pct-both-settings
IN premise — summaries/2026/08/24/xie-2024-chameleon-sloth-sA-appendix-chunk-1.md
Created 2026-08-25T02:58:59+00:00
Llama2-7B assigns >95% normalized token probability to its chosen answer in both single-source (counter-answer) and multi-source (memory-answer) settings.
Summary
Llama2-7B picks its answer tokens with over 95% probability in every tested setup, meaning it is almost never genuinely uncertain about what word comes next regardless of whether it is answering a counter-example or drawing from memory. For the system, this matters because the model's own confidence signal is essentially saturated and will not help distinguish correct from incorrect outputs, so downstream truth-tracking has to rely on external verification rather than the model's internal probability estimates.