Back To Course Home

Optimization for Machine Learning with NumPy and SciPy

0% completed

Introduction to Optimization

What Is This Course About?The Motivation for Optimization Random Search Optimization Grid Search Optimization The Nelder-Mead Algorithm Quiz Yourself on Optimization Challenge: Grid Search Solution: Grid Search

Vector Calculus

Differentiation of the Univariate Functions Partial Derivatives and Gradients Gradients of the Vector-Valued Functions Gradients of Matrices Higher-Order Gradients: Hessian Linearization and the Taylor Series Integrals and the Trapezoidal Rule Quiz Yourself on Vector Calculus Challenge: The Taylor Series Solution: The Taylor Series

Convex Optimization

Convex Sets and Functions Gradient-Solving Approach Solving Ridge Regression Fisher Discriminant Analysis Maximum Likelihood Estimation (MLE)Maximum A Posteriori (MAP)Alternating Least Squares Regression Gradient Descent: Logistic Regression Subgradient Descent—Lasso Regression Steepest Descent Coordinate Descent Quiz Yourself on Convex Optimization Challenge: Convex Optimization Solution: Convex Optimization

Gradient Descent for Non-Convex Optimization

Limitations of Gradient Descent

Minibatch Gradient Descent

The Nesterov Momentum

Nesterov Accelerated Gradient (NAG)

Adaptive Gradient Descent

Root Mean Square Propagation (RMSProp)

Adaptive Moment Estimation (Adam)

Quiz Yourself on Non-Convex Optimization

Challenge: Non-Convex Optimization

Solution: Non-Convex Optimization

Constrained Optimization

Linear Programming (LP): Formulation The Simplex Tableau Method Quadratic Programming (QP)Lagrange Duality The Dual of Linear Programming (LP)The Dual of Quadratic Programing (QP)Integer Linear Programming (ILP)Quiz Yourself on Constrained Optimization Challenge: Constrained Optimization Solution: Constrained Optimization

Miscellaneous Methods

Newton’s Method Quasi-Newton Methods Conjugate Gradient Descent The Log Barrier Method SciPy Utilities Interpolation and Curve Fitting Quiz Yourself on Miscellaneous Methods Challenge: The Log Barrier Method Solution: The Log Barrier Method

Course Conclusion

Course Assessment

Test Your Concepts of Optimization

Mini Project

Training Support Vector Machines (SVMs)

Quiz Yourself on Non-Convex Optimization

Evaluate your non-convex optimization concepts via an MCQ quiz.

1

Which optimization algorithm uses the following update rule?

v_t = \beta v_{t-1} - \alpha \nabla_\theta J(\theta_{t-1} + \beta v_{t-1}))

\theta_t = \theta_{t-1} + v_t.

A)

The Nesterov momentum

B)

Adam

C)

NAG

D)

RMSProp

Question 1 of 50 attempted

Get hands-on with 1400+ tech skills courses.