svm-grounded-in-learning-theory-but-theory-itself-overturned

IN derived (depth 2)

Created 2026-06-21T14:03:57+00:00 · Reviewed 2026-06-21T15:37:01+00:00

SVMs are uniquely grounded in statistical learning theory (PAC framework, ERM with hinge loss recovering Bayes-optimal classifier, SRM regularization) — yet the classical generalization theory that grounds them has itself been overturned by double descent and benign overfitting, creating a paradox where SVMs' strongest theoretical virtue (principled learning-theoretic foundation) rests on a framework now known to be incomplete.

Justifications

SL — SVMs are the paradigmatic application of PAC/ERM/SRM, but classical generalization theory's revision undermines the very framework that made SVMs theoretically exemplary.

Antecedents (all must be IN):

IN pac-learning-framework — PAC (Probably Approximately Correct) learning provides the mathematical/statistical framework for describing machine learning; most traditional ML and deep learning algorithms can be described as empirical risk minimization under PAC
IN svm-equivalent-erm-hinge-loss-tikhonov — The soft-margin SVM is equivalent to empirical risk minimization (ERM) with hinge loss and Tikhonov (L2) regularization.
IN classical-generalization-theory-overturned — Classical generalization theory — the U-shaped bias-variance tradeoff — has been overturned by two empirical phenomena: double descent shows test error decreasing again far past the interpolation threshold, and benign overfitting shows perfect training fit coexisting with good generalization in overparameterized regimes.