sinusoidal-positional-encoding-n-10000

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-3.md

Created 2026-06-21T09:55:55+00:00

The original transformer's sinusoidal positional encoding uses N=10000, with the key property that position shifts can be represented as linear transformations.