zhou-small-models-overabstain-opinion-prompts

IN premise — summaries/2026/08/24/zhou-2023-context-faithful-prompting-s003-the-left-column-of-the-table-presents-a-knowl.md

Created 2026-08-25T02:59:09+00:00

Smaller LLMs (≤6.7B parameters) show worse selective prediction with opinion-based prompts because they lack sufficient reading-comprehension ability and misclassify answerable questions as 'I don't know.'

Summary

When you ask a small language model (up to around 6.7 billion parameters) an opinion-style question, it tends to say "I don't know" even when it could have given a reasonable answer. In practice, this means small models are too timid to engage with subjective or preference-based prompts, so a system relying on them loses a lot of useful responses simply because the model can't fully parse what's being asked.