llava-only-linear-projection-layer-finetuned
IN premise — summaries/2026/08/24/wiki-Transformer_deep_learning_architecture-chunk-4-chunk-2.md
Created 2026-08-24T17:11:25+00:00
In the LLaVA architecture, the ViT-L/14 vision encoder and Vicuna-13B language model are bridged by a single linear projection layer, and only that projection layer is finetuned while both backbone models remain frozen
Summary
LLaVA connects its image encoder to its language model with just one simple linear transformation, and training it means adjusting only that tiny bridge while leaving both large models completely untouched. This makes fine-tuning cheap and fast, but it also caps how much adaptation can happen, since all the visual-to-linguistic alignment work is forced through a single layer.