interpretability-inversely-correlated-with-capability

IN derived (depth 1)

Created 2026-06-21T11:27:21+00:00 · Reviewed 2026-06-21T15:37:01+00:00

There is a tension between interpretability and model complexity in ML: easily interpretable model families (decision trees, linear models, rule-based models, attention-based models) tend to be simpler, while neural networks that achieve strong performance are 'black box' models requiring separate XAI research to explain. Even within a single family, scaling from a single decision tree to a random forest ensemble trades interpretability for accuracy.

Justifications

SL — The four interpretable families are precisely the classical models that don't scale, while scalable architectures are black boxes — interpretability tracks inversely with the capability frontier

Antecedents (all must be IN):

IN nn-black-box-motivates-xai — Neural networks are 'black box' models lacking interpretability, which motivates research in Explainable AI (XAI) and hybrid neural-symbolic approaches.
IN four-interpretable-model-families — The four families of easily interpretable ML models are decision trees, linear models, rule-based models, and attention-based models.
IN rf-primary-disadvantage-interpretability — The primary disadvantage of random forests compared to single decision trees is loss of interpretability, not loss of accuracy.

Dependents

These beliefs depend on this one:

OUT bias-detection-achievable-via-interpretable-models — Systemic algorithmic bias would be detectable and correctable through interpretable model families — rule-based and tree-based methods' transparent decision boundaries enable direct auditing for the discriminatory patterns documented across decades of ML deployment.
IN born-again-trees-prove-interpretability-extractable-but-not-scalable — Born-again decision trees demonstrate that interpretability can be extracted from black-box ensembles — yet this extraction path leads back to the interpretable model families whose inverse correlation with capability is already established, proving that interpretability recovery is possible in principle but constrained to the same capability ceiling that makes interpretable models insufficient.
IN capability-scaling-compounds-diagnosis-crisis — As ML capabilities scale, the crisis compounds along two independent axes simultaneously — safety mechanisms remain comprehensively absent at every level, AND interpretability decreases with increasing capability, meaning the most powerful models are simultaneously the hardest to audit and the least protected by existing defenses.
IN deployment-accountability-gap — ML faces a deployment accountability gap — algorithmic bias is systemic and documented across decades, yet the most capable deployed models are precisely those least interpretable, making bias detection and correction harder exactly where the stakes are highest.
IN ensemble-principle-spans-interpretability-capability-divide — The ensemble principle appears to span both the classical-deep ML divide and the interpretability-capability tension — it operates as explicit, interpretable variance reduction in random forests and as implicit regularization (dropout) in neural networks. This makes it a notable example of an ML mechanism that functions across both interpretable-classical and opaque-deep contexts, though whether it is unique in this regard is not established by the available evidence.
OUT interpretability-gap-closable-if-crisis-not-constitutive — The systematic inverse correlation between interpretability and capability would be closable through XAI research — since the deployment accountability gap is well-characterized and the inverse correlation motivates active research, systematic investment in interpretability could progressively narrow the gap and restore accountability for deployed ML systems.
OUT symbolic-hybrid-rescues-deployment-from-hallucination-failure — Hybrid neuro-symbolic architectures would resolve neural networks' hallucination failure mode by incorporating symbolic AI's demonstrated immunity to false-content generation — addressing one of the two complementary neural failure classes without requiring full model replacement, and breaking the compounding dynamic where hallucination and adversarial vulnerability reinforce each other unobserved.