decoder-only-has-2-sublayers-no-cross-attention

Summary

Because a decoder-only model has no separate encoder sequence to reference, each layer drops the cross-attention step entirely, leaving just two components: a self-attention pass over the sequence itself and a feed-forward computation. This makes the architecture leaner and means the model must carry all its context within its own sequence rather than pulling from an external source, which shapes how information flows and is limited in these systems.