What Is This Course About?

In applied machine learning—given a dataset—the first step is constructing an objective function to measure the error between the predicted and ground-truth scores. Once we have the objective function, the second step involves using an optimization algorithm to find a good set of parameters that minimize the objective function.

Whenever our model trains on the GPU, there is an optimization algorithm in action behind the scenes. Therefore, it is important to understand optimization to use machine learning to its full potential. Gradient descent is one of the famous algorithms used to optimize machine learning models.

In this course, we will cover optimization for machine learning. We will start with the fundamentals of optimization, such as gradients, Hessians, and convexity, and dive into some of the famous and widely used optimization algorithms, such as maxima-minima, gradient descent, duality, second-order methods, etc. We will also cover several applications of optimization, such as regression, regularized regression, maximum likelihood estimation (MLE), etc. The hands-on project at the end of the course will be based on the NumPy and SciPy libraries.

Who is this course for?

Prerequisites

This course requires learners to have a basic understanding of the following topics:

Learning outcomes

After taking this course, you will have learned the following concepts:

• The importance and need of optimization algorithms in machine learning

• The various types of optimization problems that exist and the algorithms used to solve them

• The fundamentals of optimization, such as convexity, gradients, Hessians, etc

• The implementation of popular optimization algorithms, such as gradient descent, maxima and minima, primal-dual theorem, Newtonian methods, genetic algorithms, etc

• Optimization in popular machine learning algorithms, like linear regression, regularized regression, MLE, etc