formal-learning-definition-contains-seeds-of-crisis
IN derived (depth 5)
Created 2026-06-21T13:48:35+00:00 · Reviewed 2026-06-21T15:37:01+00:00
Mitchell's formal definition of learning — improvement on task T via experience E measured by performance P — relies on a performance measure P that functions as a proxy. Since standard evaluation methodologies and overfitting defenses address training-test generalization but are not designed to detect deployment-specific failure modes such as adversarial vulnerability and algorithmic bias, the definition's reliance on P may leave a structural gap between what the formalism measures and what deployment requires — suggesting that some of ML's deployment challenges are connected to limitations already present in the foundational framing, not solely to particular methodological shortcomings.
Justifications
SL — Mitchell's definition makes P the arbiter of learning, but evaluation is doubly insufficient for deployment — so the formal definition structurally guarantees a gap between what ML can measure and what deployment requires, making the crisis definitional.
Antecedents (all must be IN):
- IN ml-mitchell-1997-formal-definition — Tom Mitchell's (1997) formal definition: a computer program learns from experience E with respect to task T and performance measure P if its performance on T as measured by P improves with experience E
- IN evaluation-doubly-insufficient-for-deployment — ML's standard evaluation methodologies (holdout, k-fold, bootstrap) and standard overfitting defenses (dropout, regularization, feature selection) address complementary aspects of the training-test generalization gap, but neither evaluation nor defense mechanisms are designed to detect deployment-specific failure modes such as adversarial vulnerability and algorithmic bias — suggesting a structural gap between training-time quality assurance and deployment-time safety.
Dependents
These beliefs depend on this one:
- IN crisis-universal-across-domains-and-embedded-in-definition — ML's crisis is simultaneously universal across application domains (validated by independent CV-NLP convergence on the same pragmatism-crisis dynamic despite opposite data modalities and traditions) AND embedded in the field's foundational formalism (Mitchell's learning definition structurally guarantees a gap between measurable and deployable performance) — establishing that the crisis is both empirically inescapable across all domains and formally inescapable from ML's own self-definition.
- IN reliability-gap-has-three-independent-impossibility-proofs — ML's reliability gap is supported by three largely independent lines of evidence operating at different levels: formal (Mitchell's definition structurally embeds the evaluation gap through proxy performance measures), economic (mathematical quality is orthogonal to evolutionary success, so reliability improvements may not survive paradigm selection), and epistemic (the crisis may be deeply intertwined with capable ML itself, suggesting reliability cannot be straightforwardly added without affecting capability) — each providing substantial independent support, collectively suggesting the gap's persistence as a structural feature rather than a solvable deficiency.