Learn to solve optimization problems in Python using essential math tools, metaheuristic methods, and constrained optimization techniques. Master efficient solutions for real-world applications.

Optimization with Python from zero to hero.png

cvxpy.tar.gz

cvxpy

Optimization theory seeks the best solution, which is pivotal for machine learning, cost-cutting in manufacturing, refining logistics, and boosting finance profits.

This course provides a detailed description of different optimization problems and the techniques used to solve them. You’ll begin with the formal definition of an optimization problem and an overview of essential mathematical tools: derivatives, gradients, and Hessian. With this knowledge, you’ll implement solutions for several optimization problems. You’ll also learn approximate metaheuristic population methods like particle swarm optimization and genetic algorithms, and solve constrained optimization problems using techniques like penalty methods and constraint simplification. Finally, you’ll solve linear programs and integer programs. 

By the end of this course, you’ll become proficient in formulating different kinds of problems as optimization problems and become skilled in the Python tools that efficiently solve these problems.

Mastering Optimization with Python

## Problem 1

$$
\min_x x^4
$$

The first problem is to find the minimum of $x^4$. This function is very similar to  $x^2$ since they both are always greater than or equal to zero. Any number powered by an even exponent will be positive. But also, they're equal to zero when $x = 0$. If the function can never be less than zero but we know a point when it's equal to zero, then that point should be the minimum of the function! So the answer is $x = 0$.

### Problem 2
$$
\min_x x^4  + 6x^2 + 1
$$
The second exercise is a little trickier. Now the function is more complex: $x^4  + 6x^2 + 1$. But don't be afraid! Let's plot this new function.

import matplotlib.pyplot as plt

def f(x):
  return x**4 + 6*x**2 + 1

inputs = list(range(-10, 11))
outputs = list(map(f, inputs))

plt.plot(inputs, outputs, '.')

# !!!: This line is used here for technical purposes regarding Educative environment
# you will use 'plt.show()' in your computer instead
plt.savefig("output/image.png") # plt.show()

As we can see, this function is still very similar to $x^2$. The minimum of the function is $x = 0$ again. The only difference is that the value of the function at that point is one, not zero. So the changes concerning the previous code are:

* **Line 4:** We change the function $f$. Now it's $x^4 + 6x^2 + 1$.

### Problems 3 and 4

$$
\max_x x \\
s.t.: x \in [-10, 10]
$$
$$
\max_x x \\
s.t.: x \in (-10, 10)
$$

The other two examples are very similar, but there's a subtle difference. In both cases, the function to be maximized is $x$ itself. The task seems easy because we just need to find the largest possible value that $x$ can have. Without constraint, the maximum wouldn't exist since we could always find a bigger value for $x$. But in both cases, we have a constraint on $x$.

Learn how to solve the optimization problems of the previous lesson.

Introduction

Derivatives and Gradients

First Optimization Algorithms

Population Methods

Adding Constraints

Solving Sudoku and the 8-Queens Puzzle as Constraint Satisfaction

Linear Constrained Optimization

Summary and Conclusion

Appendix

Maze Solver Using the Ant Colony Optimization Algorithm

Solutions to Basic Optimization Problems

Exercise 1

Problem 1

Problem 2