Tree-Structured Parzen Estimator Method Using KNN

In this example, we’ll use the KNN algorithm to determine which combination of hyperparameter values will produce the best results in comparison to the results obtained by using the default values for the hyperparameters.

What will we learn?

In this lesson, we’ll learn how to do the following things in Jupyter Notebook:

• Create and train the KNN model.

• Measure the performance of the ML model.

• Perform the steps required to implement the TPE method.

• Identify the combination of hyperparameters that provide the best results.

Import important packages

First, we import the important Python packages that will do the following tasks:

• Create and train the KNN model from scikit-learn.

• Check the ML model’s performance using the F1 score from scikit-learn.

• Implement the TPE method from the Optuna library.

• Identify the combination of hyperparameters that provide the best results using attributes from Optuna.