rag-supervised-kl-divergence
IN premise — summaries/2026/08/24/wiki-Retrieval-augmented_generation-chunk-1.md
Created 2026-08-25T02:58:51+00:00
Supervised retriever optimization minimizes KL divergence between the retriever's selection distribution and the generator's likelihood using perplexity scoring.
Summary
The retriever is trained so that the documents it picks are the same ones the language model would score as most natural and coherent, measured by how surprised the model is by each candidate. This closes the gap between the two pipeline stages, so the generator isn't handed chunks it would have rated low anyway.