What is a dual cone in convex optimization?

In this answer, we'll talk about dual cones. It's necessary to define the theorems that form the basics of dual cones; the separating hyperplane theorem and the supporting hyperplane theorem.

Basic theorems

These two theorems make up the fundamentals of dual cones.

The separating hyperplane theorem

The separating hyperplane theorem states; to make a line (hyperplane) that divides a space into two equal and opposite spaces. By opposite, we mean that the product of points with this hyperplane, in both separate spaces, shall be lesser and more significant than a $\real$ (real) number. Let's take a look at a diagram below:

So, let's say that $G$ and $S$, our two sets are disjoint, as can be seen, and there is a hyperplane that exists called $A$ which has the coordinates $(2,0)$. So we can say that there exists some $\real$ number $b$, from which the product of the transpose of A with points from both sets and $S$ are in opposition:

In our diagram above, we can take $a^T$ to be the matrix $\begin{bmatrix} 2 \\0 \end{bmatrix}$. If we take two points; $(-2,4)$ from $G$and $(3,4)$ from $S$, it will produce:

And if we take $b$ as $2$, our condition is fulfilled:

So, now, let's go ahead and understand the supporting hyperplane theorem.

The supporting hyperplane theorem

The supporting hyperplane theorem leads in from the former theorem (separation hyperplane theorem) and states two primary conditions:

Condition # 1 is simple; the hyperplane should contain at least one boundary point of the set $C$.

Condition # 2 says that the set $C$ should be enclosed in one of the two half-spaces. This is all explained in the diagram below:

The hyperplane that contains the overlapping boundary point ($x_o$) is supposed to have a linear function called y that fulfills the following condition:

The hyperplane would hence contain all the points in the Vector Space ($R$) where these points, when mapped by our linear function, would be equal to the output of $x_0$ by the linear function.

So, a statement can be issued as follows:

Now it's time to take on dual cones and understand how they work.

The dual cone

So what is the dual cone? Well, a dual cone is a cone that has points which, when multiplied by the matrix transpose of the hyperplane, breed a result greater or equal to 0 always:

If we say that we now have two supporting hyperplanes crossing the overlapping point of the cone, here it is 0, then a dual cone would come out as follows:

The dual cone D hence lies between two hyperplanes and can be defined as follows:

And you can now see that our dual cone is closed between the two hyperplanes. Plus, condition 1 is fulfilled of all points being greater than 0 by the linear function mapping.

A dual of a dual cone would again be the set $S$, or the points in $S$. One of the fundamental properties of a dual cone is that it is always convex even if the original set isn't.

In contrast to this, we've got something known as the polar cone. A mapping of a possible polar cone has been drawn below so that you can understand dual cone even better.