Feature Importance

Learn how the random forest algorithm determines the most important features for making accurate predictions.

Finding the features that matter

When using machine learning, it’s natural to ask, “Which features are the most important for making accurate predictions?” The random forest implements permutation importance to help answer this question. Permutation importance works by randomly shuffling (permuting) feature data and assessing the impact of the shuffling on the quality of predictions.

Here’s the intuition of permutation importance:

  • If you permute the values of highly predictive features, tree accuracy should decrease a lot.

  • If you permute the values of features that aren’t predictive, tree accuracy shouldn’t decrease much. ...