Support Vector Machines

Support Vector Machines

Support Vector Machines are one of the most widely used classification algorithms in Machine Learning. They are also used for Regression problems, and we have already seen their implementation in the previous lessons.

• If the data is linearly separable (meaning it needs a hyperplane to separate the classes), then the Support Vector Machine (SVM) is simple, and it finds the decision boundary which is the most distant from the points nearest to the said decision boundary from both classes.

• If the data is not linearly separable (non-linear), then Kernel Trick is used in SVM, which involves mapping the feature space in the current dimension to higher dimensions such that they are easily separable using a decision boundary. One of the benefits of Support Vector Machines is that they work very well in cases with limited datasets.

• Data points closer to the hyperplane that influence the position and orientation of the hyperplane are called Support vectors.

• SVM classification is robust to outliers.

Mathematical intuition

From Logistic Regression, we know that:

---

$y$ is the actual label of the instance.

$\hat{y} = w_0 * x_0 + w_1 * x_1 + w_2 * x_2 + w_3 * x_3 + w_4 * x_4$ ...