Linear Classifiers

Explore linear classifiers, their principles, and their training process.

What is a linear classifier?

Suppose we want to build a machine learning model to classify the following points into two categories based on their color. It is very easy to see that we can find a single point that separates them perfectly. The goal of our model is to find this point.

The easiest way to do that is to build a linear classifier. Our classifier has the form f(x,w)=w1x1+w2f(x, w) = w_1*x_1+w_2. The purpose of f(x,w)f(x,w) will be to find the parameters w1w_1 and w2w_2, so that any corresponding scalar point (1D) can be distinguished perfectly. If f(x,w)>0f(x,w)>0, the point belongs to the blue category. Otherwise, it belongs to the red. Sounds easy?

Let’s extend this idea to 2D data points!

Each point will now be represented as (x1,x2)(x_1, x_2).

For the 2D case, we need to find a line (instead of a point) that separates our 2D points, so our classifier will be f(x,w)=w1x1+w2x2+w3f(x,w) = w_1*x_1 +w_2*x_2+ w_3 ...