cad-training-free-inference-time
IN premise — summaries/2026/08/24/shi-2024-context-aware-decoding-s0-abstract.md
Created 2026-08-25T02:58:34+00:00
Context-Aware Decoding (CAD) requires no fine-tuning or parameter updates; it modifies only the decoding/sampling distribution at inference time on pretrained models.
Summary
Instead of retraining a model from scratch, Context-Aware Decoding simply nudges the model's word-by-word selection process as it generates a response. This makes it a lightweight, drop-in improvement that can be applied to any existing pretrained model without expensive compute or long training runs, lowering the barrier to getting better outputs.