rf-impurity-importance-biased-high-cardinality

IN premise — entries/2026/06/21/wiki-Random_forest-chunk-5.md

Created 2026-06-21T09:55:52+00:00

Default impurity-based feature importance in random forests is biased toward features with more categories or higher cardinality; permutation importance is the recommended unbiased alternative.

Dependents

These beliefs depend on this one:

• OUT random-forest-feature-importance-reliable — Random forest feature importance is a reliable indicator of variable relevance, supported by robust generalization properties and built-in OOB validation without needing a separate test set.
• OUT rf-impurity-importance-trustworthy — Default impurity-based feature importance in random forests is a trustworthy indicator of variable relevance, backed by OOB validation.