Identifying Support Vectors

In this lesson, we focus on support vectors, the most critical points in a Support Vector Machine. Support vectors are the data points that lie closest to the hyperplane and are responsible for defining the decision boundary and the margin between classes. Unlike other data points, support vectors directly influence the classification outcome. We’ll see how to identify them both mathematically, using the SVM equality constraint, and visually, by highlighting them on a plot. Understanding support vectors is essential because they determine the optimal hyperplane and are the backbone of the SVM algorithm.

Support vectors

Support vectors are the data points from each class that are the closest to the hyperplane. They’re responsible for defining the decision boundary or the hyperplane and, ultimately, the data classification. All other data points that aren’t support vectors do not affect the decision boundary or the hyperplane and do not contribute to the data classification.

The blue dashed line in the above graph represents the hyperplane, and the black circles with no fill are the support vectors, as these are the closest points to the hyperplane. These support vectors are the most important points in the SVM algorithm because they define the decision boundary and the margin between the two classes. Further, the plot also shows that the support vectors are few in number and lie at the boundary of the two classes. These points are crucial for determining the optimal hyperplane that separates the two classes with maximum margin. Removing or changing these points can alter the hyperplane and the classification decision boundary.

Identifying support vectors

All the support vectors $(\bold x_i,y_i)$ ...