nn-black-box-motivates-xai

IN premise — entries/2026/06/21/wiki-Neural_network_28machine_learning29-chunk-6.md

Created 2026-06-21T09:55:52+00:00

Neural networks are 'black box' models lacking interpretability, which motivates research in Explainable AI (XAI) and hybrid neural-symbolic approaches.

Dependents

These beliefs depend on this one:

• IN interpretability-inversely-correlated-with-capability — There is a tension between interpretability and model complexity in ML: easily interpretable model families (decision trees, linear models, rule-based models, attention-based models) tend to be simpler, while neural networks that achieve strong performance are 'black box' models requiring separate XAI research to explain. Even within a single family, scaling from a single decision tree to a random forest ensemble trades interpretability for accuracy.
• IN ml-two-cultures-reflected-in-architecture-divide — Breiman's two cultures (data-modeling vs. algorithmic-modeling) find a partial parallel in the classical-deep learning divide — SVMs and random forests exemplify aspects of the data-modeling culture (convex optimization, mathematical guarantees, interpretable structure), while deep neural networks exemplify aspects of the algorithmic-modeling culture (black-box prediction, hierarchical feature learning at scale), though this mapping is approximate rather than exact, and the trade-off between theoretical guarantees and empirical scaling remains an active tension rather than a settled trajectory.