xie-2021-tv-distance-bound-assumption-3

IN premise — summaries/2026/08/24/xie-2021-icl-bayesian-s2-given-hstart.md

Created 2026-08-25T02:58:55+00:00

Assumption 3 in Xie et al. (2021) bounds distribution shift via total variation: max TV(p_prompt(h) ‖ p(h|h_delim, θ*)) < Δ/4, where Δ is the margin between the most and second-most likely labels.

Summary

This assumption says that the slight difference in how a prompt-conditioned model distributes probability across labels compared to the true model is guaranteed to be smaller than one quarter of the gap between the top two candidate answers. In practice, that means the prompt-driven perturbation is too small to ever flip which label ranks first, so the system's best guess stays stable even under that mild distribution shift.