Mastering Hyperparameter Optimization for Machine Learning/

...

/

Sequential Model-Based Optimization using K-Nearest Neighbors

Sequential Model-Based Optimization using K-Nearest Neighbors

Learn to apply the sequential model-based optimization method (SMBO) to find the best hyperparameters for a K-nearest neighbors model.

We'll cover the following...

This example shows how we can use the SMBO method to optimize the hyperparameters of the ML model.

In this example, we’ll use the k-nearest neighbors (KNN) algorithm to determine which combination of hyperparameter values will produce the best results compared to the results obtained by using the default values for the hyperparameters.

What will we learn?

In this lesson, we’ll learn how to do the following things in the Jupyter Notebook:

  • Create and train an ML model (KNN).

  • Measure the performance of the ML model.

  • Perform the steps required to implement the SMBO method.

  • Identify the combination of hyperparameters that provide the best results.

Import the important packages

First, we import the important Python packages that will perform the following tasks:

  • Create and train an ML model (KNN algorithm).

  • Check ML model performance.

  • Implement the SMBO method.

  • Identify a combination of hyperparameters that provide the best results.

Press + to interact
# import important modules
import numpy as np
import pandas as pd
# sklearn modules
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import f1_score
from sklearn.model_selection import cross_val_score
from sklearn.preprocessing import MinMaxScaler
import warnings
warnings.filterwarnings("ignore")
# seeding
np.random.seed(123)

Note: The procedure for dataset preparation has been explained in detail in the ...