cad-model-size-benefit-knowledge-conflict
IN premise — summaries/2026/08/24/shi-2024-context-aware-decoding-s5-related-work.md
Created 2026-08-25T02:58:36+00:00
On knowledge-conflict benchmarks (Memotrap, NQSWAP), CAD's improvement increases with model size because larger models have a stronger tendency to rely on parametric prior knowledge.
Summary
CAD is a method for helping models resolve conflicts between what they remember from training and what new information tells them. The finding is that CAD delivers bigger improvements on larger models, because bigger models cling harder to their built-in knowledge, making the technique most valuable exactly where the stubbornness problem is worst.