Back To Course Home

Optimization for Machine Learning with NumPy and SciPy

0% completed

Introduction to Optimization

What Is This Course About?

The Motivation for Optimization

Random Search Optimization

Grid Search Optimization

The Nelder-Mead Algorithm

Quiz Yourself on Optimization

Challenge: Grid Search

Solution: Grid Search

Vector Calculus

Differentiation of the Univariate Functions Partial Derivatives and Gradients Gradients of the Vector-Valued Functions Gradients of Matrices Higher-Order Gradients: Hessian Linearization and the Taylor Series Integrals and the Trapezoidal Rule Quiz Yourself on Vector Calculus Challenge: The Taylor Series Solution: The Taylor Series

Convex Optimization

Convex Sets and Functions Gradient-Solving Approach Solving Ridge Regression Fisher Discriminant Analysis Maximum Likelihood Estimation (MLE)Maximum A Posteriori (MAP)Alternating Least Squares Regression Gradient Descent: Logistic Regression Subgradient Descent—Lasso Regression Steepest Descent Coordinate Descent Quiz Yourself on Convex Optimization Challenge: Convex Optimization Solution: Convex Optimization

Gradient Descent for Non-Convex Optimization

Limitations of Gradient Descent Minibatch Gradient Descent The Nesterov Momentum Nesterov Accelerated Gradient (NAG)Adaptive Gradient Descent Root Mean Square Propagation (RMSProp)Adaptive Moment Estimation (Adam)Quiz Yourself on Non-Convex Optimization Challenge: Non-Convex Optimization Solution: Non-Convex Optimization

Constrained Optimization

Linear Programming (LP): Formulation The Simplex Tableau Method Quadratic Programming (QP)Lagrange Duality The Dual of Linear Programming (LP)The Dual of Quadratic Programing (QP)Integer Linear Programming (ILP)Quiz Yourself on Constrained Optimization Challenge: Constrained Optimization Solution: Constrained Optimization

Miscellaneous Methods

Newton’s Method Quasi-Newton Methods Conjugate Gradient Descent The Log Barrier Method SciPy Utilities Interpolation and Curve Fitting Quiz Yourself on Miscellaneous Methods Challenge: The Log Barrier Method Solution: The Log Barrier Method

Course Conclusion

Course Assessment

Test Your Concepts of Optimization

Mini Project

Training Support Vector Machines (SVMs)

Quiz Yourself on Optimization

Evaluate yourself on the concept learned in this chapter.

1

Which category does the following optimization problem fall under?

Maximize: $40x + 50y$
Subject to: $2x + y \leq 16$

$x + y \leq 9$

$x + 2y \leq 14$

$x \geq0, y\geq 0$

$x, y \in \R$

A)

Integer programming

B)

Linear programming (LP)

C)

Quadratic programming (QP)

D)

Convex optimization

Question 1 of 50 attempted

Get hands-on with 1200+ tech skills courses.