rope-relative-position-rotation

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-5.md

Created 2026-06-21T09:55:55+00:00

RoPE (Rotary Positional Embedding) encodes position by rotating pairs of embedding dimensions by angle mθ; dot products depend only on relative position, not absolute.