bert-masked-lm-gpt-causal-lm-same-year-2018

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-7.md

Created 2026-06-21T09:55:55+00:00

BERT (2018) uses masked language modeling (bidirectional) while GPT (2018) uses causal language modeling (autoregressive left-to-right); both published the same year with opposite pre-training strategies

Dependents

These beliefs depend on this one:

• IN bert-gpt-divergence-from-common-ancestor — BERT and GPT represent a speciation event from a common ancestor: both derived from the 2017 Transformer in the same year (2018), but chose opposite pretraining strategies (masked vs. causal LM) and opposite architectural halves (encoder vs. decoder), yielding fundamentally different capability profiles.