feature-engineering-persistence-reflects-theory-incompleteness
IN derived (depth 6)
Created 2026-06-21T11:35:09+00:00 · Reviewed 2026-06-21T15:37:01+00:00
The persistence of manual feature engineering despite deep learning's partial automation suggests that ML's surviving theoretical anchor — the manifold hypothesis — may share a similar incompleteness: just as representation learning reduces but does not eliminate the need for human-engineered features (particularly in structured and tabular domains), the manifold hypothesis provides foundational architectural guidance but may not fully characterize the structure of all data encountered in practice.
Justifications
SL — Feature engineering's incomplete automation is a practical symptom of manifold theory's incompleteness as the sole theoretical foundation
Antecedents (all must be IN):
- IN feature-engineering-automation-incomplete-revolution — ML's automation of feature engineering is an incomplete revolution — deep learning reduces but does not eliminate manual feature engineering, automated methods (DFS) can outperform most human teams but not all, and production systems still require centralized feature stores for managing features across training and inference, suggesting that full representation learning has not eliminated the need for human-engineered features in structured and tabular domains.
- IN manifold-geometry-only-surviving-theoretical-anchor — The manifold hypothesis stands out as a relatively robust theoretical anchor in ML — it provides a non-biological foundation spanning the full architecture spectrum, while much of ML's broader theoretical apparatus (generalization theory, paradigm taxonomy, practical-theoretical alignment) remains in a weakened or revisionary state. This makes manifold geometry a comparatively strong candidate for principled reasoning about architecture design, though the overall theoretical landscape's instability means even this foundation should be held with appropriate uncertainty.
Dependents
These beliefs depend on this one:
- IN feature-engineering-canary-for-crisis — The persistence of manual feature engineering is a canary for ML's deeper crisis dynamic — it reflects not just the manifold hypothesis's incompleteness as a practical guide but the broader pattern where pragmatism creates capabilities (deep learning's partial automation of representation) without the theoretical depth to complete them, mirroring the innovation-without-reliability pattern at the methodology level.
- IN practical-bridges-rest-on-dissolving-foundations — ML's practical workarounds for theoretical incompleteness are systematically built on dissolving foundations — transfer learning bridges paradigms but both endpoints rest on dissolving terrain (the classical taxonomy it formalizes is fragmenting, the modern pipelines it enables are empirically fragile and transient), while persistent manual feature engineering compensates for the manifold hypothesis's incompleteness but cannot address the reliability gap it reflects, revealing that ML's practical adaptations are parasitic on the very theoretical structures whose inadequacy they are trying to compensate for.