gpt3-davinci-003-popqa-longtail-19pct

Summary

GPT-3 (the davinci-003 version) gets only about 1 in 5 answers right when quizzed on the obscure, rarely-mentioned questions in the POPQA commonsense benchmark, meaning its knowledge drops off sharply once you move past well-known topics. This sets a baseline showing that the model is far from reliable on niche or long-tail factual questions, which matters for any system that depends on it to handle less mainstream queries.