attention-nmt-origin-bahdanau-2014-luong-2015

Summary

The attention mechanism that the Transformer uses to weigh input tokens was not invented in 2017; it was first proposed in 2014 by Bahdanau and colleagues and refined in 2015 by Luong and colleagues. This matters for the system because it pins down a clear intellectual lineage: the Transformer's core "self-attention" idea builds on prior work, so any attribution or novelty analysis must treat attention as an inherited component rather than a contribution of the 2017 paper alone.