mallen-2023-gpt3-zero-shot-35pct

Summary

GPT-3 can answer roughly one in three deliberately tricky common-sense questions about obscure topics correctly even without any task-specific training, which sets a baseline for how much useful knowledge a large pre-trained model already carries before any further work is invested in improving it. This matters because it tells the system what to expect as a starting point when evaluating whether additional fine-tuning or context provides a meaningful gain.