bert-classification-finetune-cls-linear-softmax

IN premise — summaries/2026/08/24/wiki-BERT_language_model-chunk-1.md

Created 2026-08-24T17:11:05+00:00

For classification fine-tuning, BERT feeds the [CLS] token's output vector to a linear-softmax layer while updating all model parameters end-to-end.

Summary

When BERT is adapted to classify text, the standard approach treats the [CLS] token as the sentence-level summary, passes it through a single linear layer and softmax, and retrains every weight in the model to fit the new task. This matters because the entire transformer is being reshaped for classification rather than just adding a small adapter on top, so training is more expensive but the model's internal representations genuinely shift to encode what is relevant for the target task.