encoder-2-sublayers-decoder-3-sublayers

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-3.md

Created 2026-06-21T09:50:11+00:00

Each encoder layer has exactly 2 sublayers (self-attention + FFN); each decoder layer has exactly 3 sublayers (masked self-attention + cross-attention + FFN).

Summary

This captures the fixed internal recipe of each transformer layer: encoder blocks run two steps (attend to the input, then pass through a dense network), while decoder blocks run three steps (attend to the output so far, attend back to the encoder's result, then pass through a dense network). It matters because this structure defines exactly how many compute stages every token goes through per layer, which pins down the model's depth, cost, and the order in which information flows from input to generated output.