gpt3-lambda-longer-examples-improve-icl
IN premise — summaries/2026/08/24/xie-2021-icl-bayesian-sR-references-chunk-2.md
Created 2026-08-25T02:58:57+00:00
In the GPT-3 LAMBADA experiment, 5 long examples (500–600 chars) outperform 5 short examples (200–300 chars), closing ~56% of the gap to 10 independent short examples, even under train/test distribution mismatch.
Summary
In the GPT-3 LAMBADA experiment, making each in-context example longer and more detailed is a more efficient use of your prompt space than simply adding more short examples, and this advantage holds up even when the examples don't perfectly match the test data. Practically, this means that when designing prompts for an LLM, investing in richer, fuller examples yields closer-to-doubling performance gains for free, rather than spending your context budget on quantity.