Feature Selection (Filter Methods)

Feature Selection

Feature or Variable Selection refers to the process of selecting features that are used in predicting the target or output. The purpose of Feature Selection is to select the features that contribute the most to output prediction. The following line from the abstract of Machine Learning Journal sums up the purpose of Feature Selection.

The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data.

Usually these benefits of Feature Selection are quoted :

• Reduces overfitting: Overfitting has been explained in the previous lessons. If the model is overfitting, reducing the number of features is one of the methods to reduce overfitting.

• Improves accuracy: Less overfitting would perform well on the unseen dataset, so it ultimately leads to the improved accuracy of the model.

• Reduces training time: A smaller number of features means that there will be less data to train and training will be faster.

Feature Selection Methods are categorized into the following methods.

Filter Methods

The Filter Methods involve selecting features based on their various statistical scores with the output column. The selection of features is independent of any Machine ...