High level motivation: black-box optimization

\begin{equation} x^{*} \in \arg\max_{x \in S} f\qty(x) \end{equation}

where \(x\) is some candidate design, \(f\) the objective, and generally we can only see evaluations of \(y = f\qty(x) + \varepsilon\).

Baysian Optimization

A basic method is to fit a Gaussian process; given:

\begin{equation} D_{t-1} = \qty {\qty(x_{i}, y_{i})}_{i=1}^{t-1} \end{equation}

The historical measurement we fit some kind of parametrized probability distribution, say, a guassian process:

\begin{equation} f\qty(x) \sim \mathcal{GP} \qty(u_{t}, k_{t}) \end{equation}

where the Gaussian process is a generator of Gaussians given input \(x\).

And then figure out the next set of candidate evaluations based on sampling the minimum of the fitted GP.

Batch Gradient Optimizations

You can sample a batch at a time, ofc.

Design Loop

1. Evaluate objective, compute utilities
2. Train a classifier for desirable/undesirable behaviors
3. Filter for input points that passes the classifier
4. Expensively sample
5. Finetune as needed
6. Go to step 1