original-paper-row-vector-convention
IN premise — summaries/2026-08-24/wiki-Transformer_deep_learning_architecture-chunk-2.md
Created 2026-08-24T17:11:25+00:00
The original 2017 Transformer paper uses the row-vector convention, writing linear layers as xW (multiply on the right) rather than Wx (multiply on the left).
Summary
When the 2017 Transformer paper writes an equation like xW, the data vector is on the left and the weight matrix is on the right, which is the row-vector convention. This matters because many textbooks and codebases assume the opposite (Wx), so anyone translating the paper's formulas into an implementation needs to know the weights are effectively transposed compared to the more common column-vector style, or the math will silently produce wrong results.