transformer-multi-head-attention-subspaces

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-1.md

Created 2026-06-21T09:55:54+00:00

Multi-head attention runs scaled dot-product attention h times in parallel with different learned projections, allowing the model to attend to information from different representation subspaces at different positions simultaneously.

Dependents

These beliefs depend on this one:

• IN transformer-architecture-unified-yet-specialized — The Transformer is simultaneously a unified architecture (all variants built on multi-head self-attention) and a specialized one (encoder-only, decoder-only, encoder-decoder serve distinct task families).