gpt3-api-costs-vanilla-vs-retrieval

IN premise — summaries/2026/08/24/mallen-2023-when-not-to-trust-sA-appendix.md

Created 2026-08-25T02:58:12+00:00

GPT-3 davinci API costs are $0.46/1k questions (vanilla), $2.80/1k (BM25-augmented), $3.08/1k (Contriever-augmented), and $3.25/1k (GenRead), totaling $275 across 14,282 questions

Summary

Adding a retrieval step to GPT-3's davinci model roughly sextuples the per-question API cost compared to asking it directly, going from about 46 cents per thousand questions to somewhere around $2.80 to $3.25 depending on which retriever you pair with it. This matters because any system planning to use retrieval-augmented GPT-3 at scale needs to budget for that multiplier on top of what vanilla inference already costs.