xie-2021-delimiter-deterministic-emission
IN premise — summaries/2026/08/24/xie-2021-icl-bayesian-s2-given-hstart.md
Created 2026-08-25T02:58:55+00:00
Assumption 1 in Xie et al. (2021) states that a subset D of hidden states deterministically emits the delimiter token: p(o_delim | h_delim, θ) = 1 for h_delim ∈ D, and 0 otherwise, so the delimiter reveals state membership in D but not which element.
Summary
The delimiter token acts as a perfect binary flag: its appearance guarantees the model was in a specific group of hidden states, but it carries no information about which exact state within that group was active. This means any downstream reasoning about what the delimiter reveals must treat it as a coarse membership test rather than a fine-grained identifier, which caps how much state information can flow through that single token.