cad-softmax-renormalization-required

IN premise — summaries/2026/08/24/shi-2024-context-aware-decoding-s2-method.md

Created 2026-08-25T02:58:35+00:00

The raw product-of-experts expression p_θ(y_t|c,x,y_

Summary

The corrected scoring formula, before renormalization, produces values that do not add up to one across all possible next tokens, so they cannot be treated as a proper probability distribution. Without applying a softmax over the full vocabulary to fix this, any downstream step that assumes valid probabilities, such as sampling, loss computation, or comparison with a target distribution, will be mathematically broken.