l2-diffuse-l1-sparse-regularization

IN premise — entries/2026/06/21/wiki-Convolutional_neural_network-chunk-4.md

Created 2026-06-21T09:55:49+00:00

L2 regularization (weight decay) penalizes squared weight magnitude and encourages diffuse weight vectors; L1 regularization produces sparse weights; elastic net combines both L1 and L2

Dependents

These beliefs depend on this one:

• IN regularization-as-bias-variance-engineering — Regularization is fundamentally bias-variance engineering — it deliberately increases bias to reduce variance, with L2 producing diffuse weight shrinkage and L1 producing sparsity, both interpretable as Bayesian priors on the weight distribution.