kv-caching-autoregressive-inference

IN premise — entries/2026/06/21/wiki-Transformer_deep_learning_architecture-chunk-5.md

Created 2026-06-21T09:55:56+00:00

KV caching reuses previously computed key and value vectors during autoregressive inference since they don't change; prefilling is the initial forward pass that populates the cache.