zhou-2023-joint-vs-pertoken-probability

IN premise — summaries/2026/08/24/zhou-2023-context-faithful-prompting-s4-experiments.md

Created 2026-08-25T02:59:11+00:00

For abstention probability calculation, joint probability works best for GPT-3.5 while per-token probability (joint normalized by length) works best for LLaMA-2.

Summary

When deciding whether a model should say "I don't know" rather than guess, the right confidence formula depends on which model you are using. GPT-3.5 is best calibrated with raw joint probability, while LLaMA-2 needs that probability adjusted for answer length, meaning a single universal abstention threshold won't work across architectures.