encoder-only-decoder-only-encoder-decoder-variants

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-8.md

Created 2026-06-21T09:55:55+00:00

BERT is encoder-only, GPT series is decoder-only, T5 is encoder-decoder — all are Transformer architecture variants

Dependents

These beliefs depend on this one:

• IN bert-gpt-divergence-from-common-ancestor — BERT and GPT represent a speciation event from a common ancestor: both derived from the 2017 Transformer in the same year (2018), but chose opposite pretraining strategies (masked vs. causal LM) and opposite architectural halves (encoder vs. decoder), yielding fundamentally different capability profiles.
• IN transformer-architecture-unified-yet-specialized — The Transformer is simultaneously a unified architecture (all variants built on multi-head self-attention) and a specialized one (encoder-only, decoder-only, encoder-decoder serve distinct task families).