bert-encoder-only-bidirectional

OUT premise — entries/2026/06/21/wiki-BERT_language_model-chunk-1.md

Created 2026-06-21T09:50:09+00:00

BERT uses only the encoder portion of the transformer with bidirectional (all-to-all) self-attention and no causal masking, making it unable to generate text