Introduction to K-Nearest Neighbors

The k-nearest neighbors (KNN) is widely used for classification and is one of the simplest machine learning algorithms. The principle behind the method is to find a predefined number of training samples closest in the distance to the test point and predict the label from these. In simple words, KNN stores the entire dataset as a training algorithm and categorizes the test data point using stored information of its nearest neighbors, where $k$ is the number of neighbors. Let’s learn with example data.

Example

Let’s consider:

• We have a dataset with two classes, A and B, and the entire data, along with its features, f1, and f2, are stored as a training algorithm.

• We want to predict the class for red, green, and blue test data points.

• Based on the association with their neighboring points, it's straightforward to predict the class for red and green.

The nearest neighboring data points for red are in class B, so KNN will predict B for red. The closest neighboring data points for green are in A, so KNN will predict class A for green. What about the blue data point? Let’s discuss KNN with another more straightforward example of fewer data points to predict the class for the blue data point. We already ...