Support Vector Machines
This lesson will focus on training Support Vector Machines.
We'll cover the following...
Support Vector Machine
A Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression problems. In this algorithm, we plot each data item as a point in n-dimensional space (where n is the number of features we have) with the value of each feature being the value of a particular coordinate. Then, we perform classification by finding the hyper-plane/line that differentiates the two classes. A hyper-plane is the higher dimensional version of a line. As a line can separate classes on two-dimensional data, a hyper-plane can separate on higher dimensions.
We can see that the line separates the two classes (green and blue) very well. These data points are called support vectors. Altering these points can alter the line. The black line is the support vector machine here.
When predicting classes, an observation is plotted, and if it falls on the left side of the line, it is classified as the green class, if it falls on the right side of the line, it is classified as the blue class.
Kernels
The example we saw above is too simple. Usually, classes are not easy to separate. Sometimes we cannot separate data easily by a line or a hyperplane. Therefore, SVMs are found by converting data into higher dimensions and finding hyper-planes on the higher dimensional data. This is known as the kernel trick. Kernels are functions that take low dimensional data and transform it into a higher dimensional space, i.e., they convert an ...