cad-distinguishes-from-dexperts
IN premise — summaries/2026/08/24/shi-2024-context-aware-decoding-sR-references.md
Created 2026-08-25T02:58:36+00:00
Unlike DExperts (which demotes a separate anti-expert model) and contrastive decoding (Li et al., 2022) (which demotes a separate small amateur model), CAD demotes the same model's own context-free prior-knowledge distribution.
Summary
CAD takes a different approach from other contrastive decoding methods by not relying on a second model at all. Instead of subtracting the output of a separate "amateur" or "anti-expert" model, it subtracts what the same model would generate with no context, so the correction targets the model's own built-in biases rather than an external proxy's behavior. This makes the method simpler to deploy and more principled, since the thing being removed is the model's genuine prior tendency rather than an approximation of one.