mqa-gqa-mla-kv-cache-variants

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-5.md

Created 2026-06-21T09:55:56+00:00

MQA shares K/V projections across all heads; GQA partitions heads into groups sharing K/V; MLA uses low-rank projection to compress KV cache to low-dimensional latent vectors.