Numeric Feature SSE

Finding the optimal split

Similar to classification trees, the CART algorithm has to find optimal regression tree splits efficiently. In the case of numeric features, this is a challenge because numeric features have many unique values. Once again, the CART algorithm utilizes an optimization to evaluate potential split points efficiently.

Consider training a CART regression tree model to impute missing values of the Age feature of the Titanic dataset. Further, one of the predictive features used to train the imputation model is the Fare feature of the Titanic dataset.

CART regression trees use a similar optimization strategy as classification trees for determining optimal splits with numeric predictive features:

1. Sort the predictive feature (i.e., Fare).

2. Evaluate potential splits only where the values of the feature change.

3. Note that split points are equidistant from the feature values.

A numeric feature example

The following is sample data from the Titanic training set. The observations in the table have been sorted based on the values of the Fare feature: