google-model-lineage-transformer-to-gemini

IN premise — entries/2026/06/21/wiki-BERT_language_model-chunk-3.md

Created 2026-06-21T09:50:09+00:00

Google's core language model evolution: Transformer (2017) → BERT (2018) → T5 (2019) → LaMDA (2021) → PaLM (2022) → Gemini (2023)

Summary

Gemini is not a standalone invention but the latest step in a direct architectural lineage stretching back to the Transformer paper in 2017, meaning its capabilities and constraints are inherited through a chain of design decisions made across several prior model generations. This matters because any strength or limitation in Gemini can be traced back to a specific architectural choice from an earlier model in the sequence, so understanding or challenging any one link in that chain has implications for the whole line.

Dependents

These beliefs depend on this one:

IN google-research-lineage-spans-transformer-to-frontier — Google's research lineage connects the original Transformer (2017), BERT (2018), Chain-of-Thought prompting (2022), and the model evolution through to Gemini — a continuous thread from foundational architecture to frontier capability.