random-forest-feature-importance-reliable

OUT derived (depth 2)

Created 2026-06-21T10:06:02+00:00

Random forest feature importance is a reliable indicator of variable relevance, supported by robust generalization properties and built-in OOB validation without needing a separate test set.

Justifications

SL — Feature importance reliability depends on absence of high-cardinality bias in the feature set

Antecedents (all must be IN):

IN random-forest-robust-generalization — Random forests achieve robust generalization through three complementary mechanisms: variance reduction via averaging decorrelated trees, immunity to overfitting from additional trees, and built-in OOB error estimation without a separate test set.
IN rf-oob-error-internal-validation — Out-of-Bag (OOB) error provides an internal estimate of random forest generalization error without needing a separate validation set, using the ~37% of samples not selected in each bootstrap.

Unless (any of these IN defeats this justification):

IN rf-impurity-importance-biased-high-cardinality — Default impurity-based feature importance in random forests is biased toward features with more categories or higher cardinality; permutation importance is the recommended unbiased alternative.