swiGLU-llama-palm-gelu-gpt-bert
IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-5.md
Created 2026-06-21T09:55:56+00:00
SwiGLU activation is used by Llama and PaLM; GELU is used by GPT-1 and BERT.
IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-5.md
Created 2026-06-21T09:55:56+00:00
SwiGLU activation is used by Llama and PaLM; GELU is used by GPT-1 and BERT.