cross-attention-queries-decoder-keys-values-encoder

IN premiseentries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-3.md

Created 2026-06-21T09:50:11+00:00

In cross-attention, queries come from the decoder and keys/values come from the encoder output.

Summary

This pins down the direction of information flow between the two halves of a transformer: the decoder is the one asking questions, and the encoder holds the reference material it looks up. It matters because it makes the encoder's output a fixed lookup table for the decoder, so the encoder never adapts to what the decoder has already produced.