svm-solver-ecosystem-scale-dependent

IN derived (depth 1)

Created 2026-06-21T10:01:28+00:00 · Reviewed 2026-06-21T15:37:01+00:00

SVMs have a rich ecosystem of specialized solvers whose optimal choice depends on problem geometry — SMO decomposes into 2D subproblems for general QP, Pegasos uses stochastic gradient descent in the primal for large-scale problems, LIBLINEAR achieves linear-time iterations for high-dimensional sparse data, and the crossover point between sub-gradient and coordinate descent depends on whether n or d dominates.

Justifications

SL — Four solver beliefs reveal that SVM optimization is not one algorithm but an ecosystem of scale-dependent strategies

Antecedents (all must be IN):

IN svm-smo-algorithm-2d-subproblems — The SMO (Sequential Minimal Optimization) algorithm decomposes the SVM quadratic programming problem into 2-dimensional sub-problems solved analytically, requiring no numerical optimizer or large matrix storage.
IN svm-pegasos-sgd-solver — Pegasos (Shalev-Shwartz et al. 2007) is a stochastic gradient descent SVM solver operating in the primal with O(1/t) convergence.
IN svm-liblinear-linear-time-iterations — LIBLINEAR's coordinate descent for linear SVMs has iterations that are linear in data read time with Q-linear convergence, making it extremely fast.
IN svm-subgradient-large-n-coorddescent-high-dim — Sub-gradient descent is preferred for SVMs with large n (many training examples); coordinate descent is preferred for high-dimensional feature spaces.

Dependents

These beliefs depend on this one:

IN svm-complexity-compounds-with-scale — SVM complexity compounds as problems scale — multiclass classification requires architectural decomposition (OvA/OvO/Crammer-Singer) on top of already scale-dependent solver selection (SMO vs Pegasos vs LIBLINEAR), creating a combinatorial methodology burden that contrasts with neural network approaches which handle multiclass classification more naturally.