Introduction to the Course

Optimization is at the core of many fields of science and engineering. Think about a practical problem and it will surely have some optimization component. For example, in the field of engineering, optimization allows us to control processes and design bridges, cars, and even cans of food. Machine learning (ML) is, from a mathematical perspective, an optimization problem. Most of the things created by humans are the way they are because of optimization. They have the optimal shape, size, and weight. They’re the optimal combination of materials and processes. They’re the optimal solution to a problem.

This course is a broad introduction to optimization. We’ll cover what optimization is and how to solve a wide variety of optimization problems. We’ll solve them not just on paper, but also code the solution with Python. So, when you finish this course, you’ll be able to optimize a lot of processes and solve a diverse array of problems using Python and sophisticated optimization techniques.

Prerequisites

This course can be helpful for lots of learners from different backgrounds. For beginners, this course is an excellent way to get started with problem-solving and algorithms. Of course, the focus is on optimization algorithms, but most of the skills you acquire here are transferable. For people trying to dive deeper into machine learning and AI, this course brings one of those missing pieces behind the scenes. Most of the algorithms in ML and AI are, in essence, solutions to an optimization problem. Understanding how optimization works will help you understand why some algorithms work better than others in some circumstances and why that neural network seems not to be learning anything at all.

Although not a programming course, the coding examples and exercises with Python we’ll see are an excellent way to sharpen your programming skills and learn a lot of useful tools available in Python to solve optimization problems. This is a course with a lot of practical examples and exercises. You’ll learn by doing.

Basic knowledge of calculus, linear algebra, and programming in Python is required. We’ll cover the basics of many mathematical concepts that are important for optimization, but you’ll get a lot more out of this course if this isn’t the first time you’re hearing about them. As long as you have a basic knowledge of Python, you’ll be able to follow the course. We’ll explain the specific code about optimization, but we assume you know basic concepts like lists, dictionaries, and functions in Python.

What will we learn?

In this introductory chapter, we’ll learn about the course and the tools you’ll need to follow it. You’ll also learn about basic optimization problems and their solutions as an appetizer for what’s to come. Chapter 2 comes with some math. There, we’ll visit derivatives and gradients, which are the basis for solving many optimization problems. Don’t worry. All is explained in detail and with lots of examples, including code.

In Chapter 3, the party begins! We’ll see the first optimization algorithms there. All those algorithms are used nowadays, so you’ll learn about valuable tools from the beginning. But be careful, great power comes with great responsibility, so the challenges you’ll find in this chapter will be more complex.

Chapter 4 opens a new world of optimization algorithms: population methods. Those algorithms don’t use as much calculus as those in Chapter 3. They bring a lot of interesting ideas and techniques to the table, based on evolution and genetics. Yeah, you read that right. But don’t worry, we won’t need our biology textbook to follow the chapter.

Chapter 5 is when constraints come in. Sometimes, theoretically possible solutions aren’t feasible. We’ll see how to deal with that here. This chapter brings the last important piece to the optimization puzzle. Then, in Chapters 6 and 7, we’ll visit linear constrained optimization and discrete optimization, respectively. These will be the most advanced, powerful, and robust algorithms we’ll see in this course.

If you get lost in some of the chapters (although we strongly recommend you revisit that chapter), in the last part of the course, you’ll find a summary of all that we’ve seen, along with recommendations for where to go next in case you want to dive deeper into optimization.

All the chapters have quizzes and practical exercises besides the code examples.

So, are you ready to start? Let’s go!