base-models-lower-accuracy-428-vs-508-percent
IN premise — summaries/2026/08/24/convergence-without-understanding-2026-sR-references.md
Created 2026-08-24T17:10:53+00:00
Base models show lower mean accuracy (42.8%) than instruction-tuned models (50.8%) on the 800-problem evaluation set, reflecting difficulty extracting structured answers without instruction-tuning.
Summary
Models that have never been trained to follow instructions score noticeably worse on structured-answer tasks than their instruction-tuned counterparts, with an 8-point accuracy gap on the 800-question set. This tells us that much of the remaining failure isn't a reasoning problem but a formatting one — the models know the answer but can't reliably pull it into the expected structure without explicit training to do so.