cross-attention-uses-decoder-query-encoder-keyvalue

IN premise — summaries/2026/08/24/wiki-Transformer_deep_learning_architecture-chunk-3.md

Created 2026-08-24T17:11:25+00:00

Cross-attention in the transformer decoder uses queries (Q) from the decoder representation and keys (K) and values (V) from the encoder output, making it the only sublayer unique to the decoder block

Summary

Cross-attention is the wiring pattern that lets the decoder look up information in the encoder's output while deciding what to generate next, using the decoder's own signal as the question and the encoder's result as the reference material. This is the one sublayer that has no counterpart in the encoder, making it the structural bridge that ties the two halves of the transformer together.