flashattention-exact-communication-avoiding

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-5.md

Created 2026-06-21T09:55:56+00:00

FlashAttention is a communication-avoiding algorithm that computes exact attention (not an approximation) using block matrix multiplications sized to fit GPU cache.

Dependents

These beliefs depend on this one:

• IN flashattention-validates-hardware-architecture-coevolution-within-paradigm — FlashAttention demonstrates that hardware-architecture co-evolution operates not only across paradigm transitions (GPU→TPU driving architecture selection) but within a single architecture — exactly matching computation to GPU cache hierarchy while preserving mathematical equivalence, showing that hardware adaptation is continuous rather than discrete.