Mastering Hyperparameter Optimization for Machine Learning/

...

/

Tree-Structured Parzen Estimator Method Using KNN

Tree-Structured Parzen Estimator Method Using KNN

Learn how to apply the Tree-Structured Parzen Estimator (TPE) method to find the best hyperparameters for a KNN model.

We'll cover the following...

In this example, we’ll use the KNN algorithm to determine which combination of hyperparameter values will produce the best results in comparison to the results obtained by using the default values for the hyperparameters.

What will we learn?

In this lesson, we’ll learn how to do the following things in Jupyter Notebook:

  • Create and train the KNN model.

  • Measure the performance of the ML model.

  • Perform the steps required to implement the TPE method.

  • Identify the combination of hyperparameters that provide the best results.

Import important packages

First, we import the important Python packages that will do the following tasks:

  • Create and train the KNN model from scikit-learn.

  • Check the ML model’s performance using the F1 score from scikit-learn.

  • Implement the TPE method from the Optuna library.

  • Identify the combination of hyperparameters that provide the best results using attributes from Optuna.

Press + to interact
# import important modules
import numpy as np
import pandas as pd
# sklearn modules
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import f1_score
from sklearn.model_selection import cross_val_score
from sklearn.preprocessing import MinMaxScaler
import warnings
warnings.filterwarnings("ignore")
# seeding
np.random.seed(123)

Note: The procedure for dataset preparation has been explained in detail in the Data Preparation lesson. Please refer to the lesson to gain insights into how the data was prepared. ... ...