Linear Support Vector Machines (SVMs)

Linear classifiers with large margins

In this lesson, we’ll outline the basic idea behind support vector machines $\text {(SVMs)}$ that have been instrumental in the first wave of industrial applications due to their robustness and ease of use.

Note: $\text {SVMs}$ have some intense mathematical underpinning, although our goal here is to outline only some of the mathematical ideas behind this method.

It is not strictly necessary to read this lesson in order to follow the rest of the course, but it does provide a summary of concepts that have been instrumental in previous progress and are likely to influence the development of further methods and research. This includes some examples of advanced optimization techniques and the idea of kernel methods. While we mention some formulae in what follows, we do not derive all the steps and will only use them to outline the form to understand why we can apply a kernel trick. Our purpose here is mainly to provide some intuitions.