rf-impurity-importance-trustworthy

OUT derived (depth 1)

Created 2026-06-21T09:59:01+00:00

Default impurity-based feature importance in random forests is a trustworthy indicator of variable relevance, backed by OOB validation.

Justifications

SL — Impurity importance is reliable when high-cardinality bias is absent

Antecedents (all must be IN):

IN random-forest-breiman-2001-formulation — Leo Breiman (2001) formulated the modern random forest by combining bagging, random feature selection at each split, out-of-bag error estimation, and permutation importance.
IN rf-oob-error-internal-validation — Out-of-Bag (OOB) error provides an internal estimate of random forest generalization error without needing a separate validation set, using the ~37% of samples not selected in each bootstrap.

Unless (any of these IN defeats this justification):

IN rf-impurity-importance-biased-high-cardinality — Default impurity-based feature importance in random forests is biased toward features with more categories or higher cardinality; permutation importance is the recommended unbiased alternative.