cross-attention-queries-decoder-keys-values-encoder
IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-3.md
Created 2026-06-21T09:50:11+00:00
In cross-attention, queries come from the decoder and keys/values come from the encoder output.
Summary
This pins down the direction of information flow between the two halves of a transformer: the decoder is the one asking questions, and the encoder holds the reference material it looks up. It matters because it makes the encoder's output a fixed lookup table for the decoder, so the encoder never adapts to what the decoder has already produced.