Black Box Bayesian Optimization

What is black box optimization?

Black box optimization (BBO) refers to a scenario in which we have a function that we want to optimize, but have little to no information about its internal structure, such as its derivatives or explicit mathematical form. This is a common situation in many real-world optimization problems, where the function might be computationally expensive or involve complex simulations.

For example, we want to know the relationship between the number of products a machine produces to the number of hours it works. We consider this an ideal state where every piece of information is known so we can create this optimization problem in a simple linear manner. This is shown below:

Now, let’s suppose we’re working with some complex relationships, and mapping all of them is computationally expensive and includes a lot of effort. So instead of doing this for each setup and completing it, it makes more sense to convert this to a BBO. Let’s see this in the illustration as below:

In such cases, traditional optimization methods that rely on gradient information or assume a specific form of the function may not be applicable. This is where Bayesian optimization comes into play as an effective approach for BBO.

How does black box optimization work?

BBO generally follows the steps below to complete the optimization process: ...