Learn to solve optimization problems in Python using essential math tools, metaheuristic methods, and constrained optimization techniques. Master efficient solutions for real-world applications.

Optimization with Python from zero to hero.png

cvxpy.tar.gz

cvxpy

Optimization theory seeks the best solution, which is pivotal for machine learning, cost-cutting in manufacturing, refining logistics, and boosting finance profits.

This course provides a detailed description of different optimization problems and the techniques used to solve them. You’ll begin with the formal definition of an optimization problem and an overview of essential mathematical tools: derivatives, gradients, and Hessian. With this knowledge, you’ll implement solutions for several optimization problems. You’ll also learn approximate metaheuristic population methods like particle swarm optimization and genetic algorithms, and solve constrained optimization problems using techniques like penalty methods and constraint simplification. Finally, you’ll solve linear programs and integer programs. 

By the end of this course, you’ll become proficient in formulating different kinds of problems as optimization problems and become skilled in the Python tools that efficiently solve these problems.

Mastering Optimization with Python

At this point, we can solve almost any optimization problem either using exact, approximate, or heuristic solutions. But we haven't learned any method to deal with constraints.

Remember the general form of an optimization algorithm:

$$
\min_x f(x) \\
s.t.: x \in C
$$

Here, $x$ can be a number, a vector, or a matrix. The second line is a constraint. It tells us that $x$ has to fulfill some properties.

# Constraints

**Constraints** are restrictions over the set of all possible solutions. When we have constraints, we can't consider all the solutions anymore; we can only consider the solutions that fulfill the constraints. This space of valid solutions is called the **feasible region**. In contrast, the set of invalid solutions is the **unfeasible region**.

It's tempting to think that constraints are good. After all, they cut the solution space and make it smaller so we need to explore fewer solutions. The reality is quite different though. Although sometimes constraints make our lives easier, most of the time they complicate things. For example, it's very difficult to obtain the feasible region and get rid of the unfeasible one. So we need to check if a solution is valid, which becomes an additional step in the algorithm and can heavily increase runtime.

Learn what constraints are and why they’re necessary.

Introduction

Derivatives and Gradients

First Optimization Algorithms

Population Methods

Adding Constraints

Solving Sudoku and the 8-Queens Puzzle as Constraint Satisfaction

Linear Constrained Optimization

Summary and Conclusion

Appendix

Maze Solver Using the Ant Colony Optimization Algorithm

Constrained Optimization

Constraints