gpt1-117m-params-bookcorpus-june-2018

IN premise — summaries/2026/08/24/wiki-Generative_pre-trained_transformer-chunk-1.md

Created 2026-08-24T17:11:10+00:00

GPT-1 (June 2018, OpenAI) had 117 million parameters and was the first decoder-only generative pre-training LLM, using BookCorpus for pre-training followed by discriminative fine-tuning.

Summary

This records the foundational GPT-1 milestone: a 117-million-parameter model from June 2018 that first proved out the "pre-train a decoder-only model to generate text, then fine-tune it for specific tasks" recipe that every modern large language model still follows. It matters to the system as a fixed historical anchor — any later claim about how LLMs work or when key architectural shifts happened traces back to this baseline.