coherence-boosting-mechanism

IN premise — summaries/2026/08/24/shi-2024-context-aware-decoding-s6-conclusion.md

Created 2026-08-25T02:58:37+00:00

Coherence boosting (Malkin et al., 2021) demotes a short-context model's distribution from the full-context distribution to emphasize long-range coherence, a related but distinct contrastive pair compared to CAD's no-context vs. full-context approach.

Summary

Coherence boosting tweaks a model's short-context predictions by pushing them away from the full-context distribution, specifically to sharpen long-range consistency. This matters because it is a different contrastive mechanism than CAD's zero-versus-full-context trick, so the two techniques should not be lumped together or assumed to produce equivalent effects.