performers-linear-attention-on-complexity
IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-6.md
Created 2026-06-21T09:50:11+00:00
Performers (Choromanski et al., 2020) use kernel-based linear attention to achieve O(n) complexity versus the standard Transformer's O(n²) self-attention.
Summary
The Performers architecture shows that by approximating the attention mechanism with simpler mathematical kernels, you can keep the same expressive power as a standard Transformer while cutting the cost from growing quadratically to growing linearly with sequence length. This matters because it means long documents, video streams, or genomic sequences become practical to process without the memory and speed wall that standard attention hits.