cad-flan-t5-nq-decrease

IN premise — summaries/2026/08/24/shi-2024-context-aware-decoding-s4-results.md

Created 2026-08-25T02:58:35+00:00

FLAN-T5 shows a minor decrease on the original (non-conflict) NQ set with CAD, attributed to NQ being part of its instruction-finetuning data.

Summary

Because the Natural Questions dataset was part of FLAN-T5's training data, its small performance drop on NQ when CAD is applied is likely just a minor side effect of the model being overly familiar with that specific set rather than a real loss of general ability. This matters because FLAN-T5 is a somewhat biased test subject for evaluating how CAD affects question-answering performance.