Home>Courses>Optimization for Machine Learning with NumPy and SciPy

Intermediate

Certificate of Completion

Optimization for Machine Learning with NumPy and SciPy

Learn optimization for machine learning, including gradients, convex optimization, and gradient descent. Explore constrained optimization and advanced methods using NumPy and SciPy.

AI-POWERED

Explanations

Adaptive Learning

AI-POWERED

Explanations

Adaptive Learning

This course includes

62 Lessons

1 Project

1 Assessment

43 Playgrounds

12 Challenges

6 Quizzes

Course Overview

What You'll Learn

Course Content

Course Overview

In this course, you will learn about optimization, one of the fundamental pillars of mathematics and machine learning. Machine learning depends heavily on optimization because it allows the model to learn from data and generate precise predictions. You will begin by introducing optimization. Then, you will learn about optimization basics, including gradients and integrals. Next, you will cover convex optimization. You will then learn how to compute gradient descent for non-convex optimization. Next, you’ll...Show More

In this course, you will learn about optimization, one of the fundamental pillars of mathematics and machine learning. Machine l...Show More

TAKEAWAY SKILLS

Optimization

Machine Learning

Plotting

What You'll Learn

An understanding of how real-world problems can be framed as optimization problems

Working knowledge about the taxonomy of optimization problems and techniques

Familiarity with the fundamental concepts of optimization

Hands-on experience implementing popular optimization algorithms

The ability to solve popular machine learning problems using optimization

An understanding of how real-world problems can be framed as optimization problems

Course Content

Introduction to Optimization

8 Lessons

Get familiar with optimization techniques, algorithms, and their applications in machine learning.

What Is This Course About?

The Motivation for Optimization

Random Search Optimization

Grid Search Optimization

The Nelder-Mead Algorithm

Quiz Yourself on Optimization

Challenge: Grid Search

Solution: Grid Search

Vector Calculus

10 Lessons

Discover the logic behind differentiation, gradients, Hessian, Taylor series, and integrals in vector calculus.

Differentiation of the Univariate Functions

Partial Derivatives and Gradients

Gradients of the Vector-Valued Functions

Gradients of Matrices

Higher-Order Gradients: Hessian

Linearization and the Taylor Series

Integrals and the Trapezoidal Rule

Quiz Yourself on Vector Calculus

Challenge: The Taylor Series

Solution: The Taylor Series

Convex Optimization

14 Lessons

Examine convex optimization techniques, including gradient methods, ridge regression, and Fisher analysis.

Convex Sets and Functions

Gradient-Solving Approach

Solving Ridge Regression

Fisher Discriminant Analysis

Maximum Likelihood Estimation (MLE)

Maximum A Posteriori (MAP)

Alternating Least Squares Regression

Gradient Descent: Logistic Regression

Subgradient Descent—Lasso Regression

Steepest Descent

Coordinate Descent

Quiz Yourself on Convex Optimization

Challenge: Convex Optimization

Solution: Convex Optimization

Gradient Descent for Non-Convex Optimization

10 Lessons

Grasp the fundamentals of advanced gradient descent techniques for optimizing non-convex problems.

Limitations of Gradient Descent

Minibatch Gradient Descent

The Nesterov Momentum

Nesterov Accelerated Gradient (NAG)

Adaptive Gradient Descent

Root Mean Square Propagation (RMSProp)

Adaptive Moment Estimation (Adam)

Quiz Yourself on Non-Convex Optimization

Challenge: Non-Convex Optimization

Solution: Non-Convex Optimization

Use Particle Swarm Optimizer to Optimize a Non-convex Function

Project

Constrained Optimization

10 Lessons

Explore constrained optimization using linear programming, simplex method, and Lagrange duality.

Linear Programming (LP): Formulation

The Simplex Tableau Method

Quadratic Programming (QP)

Lagrange Duality

The Dual of Linear Programming (LP)

The Dual of Quadratic Programing (QP)

Integer Linear Programming (ILP)

Quiz Yourself on Constrained Optimization

Challenge: Constrained Optimization

Solution: Constrained Optimization

Miscellaneous Methods

9 Lessons

Tackle diverse optimization techniques, including Newton's, Quasi-Newton, barrier methods, and SciPy utilities.

Newton’s Method

Quasi-Newton Methods

Conjugate Gradient Descent

The Log Barrier Method

SciPy Utilities

Interpolation and Curve Fitting

Quiz Yourself on Miscellaneous Methods

Challenge: The Log Barrier Method

Solution: The Log Barrier Method

Course Conclusion

1 Lessons

Build on essential optimization techniques for machine learning, enhancing models with advanced algorithms.

Conclusion

Test Your Concepts of Optimization

Assessment

Training Support Vector Machines (SVMs)

Project

Course Author

Puneet Mangla

Trusted by 2.6 million developers working at companies

"These are high-quality courses. Trust me. I own around 10 and the price is worth it for the content quality. EducativeInc came at the right time in my career. I'm understanding topics better than with any book or online video tutorial I've done. Truly made for developers. Thanks"

Anthony Walker

@_webarchitect_

"Just finished my first full #ML course: Machine learning for Software Engineers from Educative, Inc. ... Highly recommend!"

Evan Dunbar

ML Engineer

"You guys are the gold standard of crash-courses... Narrow enough that it doesn't need years of study or a full blown book to get the gist, but broad enough that an afternoon of Googling doesn't cut it."

Carlos Matias La Borde

Software Developer

"I spend my days and nights on Educative. It is indispensable. It is such a unique and reader-friendly site"

Souvik Kundu

Front-end Developer

"Your courses are simply awesome, the depth they go into and the breadth of coverage is so good that I don't have to refer to 10 different websites looking for interview topics and content."

Vinay Krishnaiah

Software Developer

"I've tried probably 5-7 different sites and Educative is easily the best. It perfectly blends explanation with interactivity"

Eric Downs

Musician/Entrepeneur

Hands-on Learning Powered by AI

See how Educative uses AI to make your learning more immersive than ever before.

Instant Code Feedback

Evaluate and debug your code with the click of a button. Get real-time feedback on test cases, including time and space complexity of your solutions.

AI-Powered Mock Interviews

Put your skills to the test in a simulated interview setting. Receive personalized feedback based on your performance. Available in Premium & Premium Plus plans.

Adaptive Learning

At various checkpoints throughout Educative courses, you will be prompted to take a quick assessment. Receive a condensed curriculum tailored to your strengths and skill gaps.

Explain with AI

Select any text within any Educative course, and get an instant explanation — without ever leaving your browser.

AI Code Mentor

AI Code Mentor helps you quickly identify errors in your code, learn from your mistakes, and nudge you in the right direction — just like a 1:1 tutor!

Free Resources

FOR TEAMS

Interested in this course for your business or team?

Unlock this course (and 1,000+ more) for your entire org with DevPath