original-transformer-relu-activation
IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-3.md
Created 2026-06-21T09:55:55+00:00
The original transformer used ReLU activation in feedforward layers.
IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-3.md
Created 2026-06-21T09:55:55+00:00
The original transformer used ReLU activation in feedforward layers.