constituents

• where \(x \in \mathbb{R}^{n}\) is a vector of variables
• \(f_{0}\) is the objective function, “soft” to be minimized
• \(f_{1} … f_{m}\) are the inequality constraints
• \(g_{1} … g_{p}\) are the equality constraints

requirements

Generally of structure:

\begin{equation} \min f_{0}\qty(x) \end{equation}

subject to:

\begin{align} f_{i} \qty(x) \leq 0, i = 1 \dots m \\ g_{i}\qty(x) = 0, i = 1 \dots p \end{align}

solving optimization problems

You can’t generally solve optimization problems… Some types

• convex optimization problems, you can solve these
• non-linear optimization

additional information

optimization terms

feasible

\(x \in \mathbb{R}^{n}\) is feasible if \(x \in \text{dom } f_{0}\) and it satisfies constraitns

optimal value

\begin{equation} p^{*} = \text{inf}\qty {f_{0}\qty(x) \mid f_{i}\qty(x) \leq 0, i = 1, \dots, m, h_{i}\qty(x) = 0, i = 1 \dots p} \end{equation}

• \(p^{*} = \infty\) if a problem is infeasible
• \(p^{*} = -\infty\) if a problem is unbounded below
• a feasible \(x\) is optimal if \(f_{0}\qty(x) = p^{*}\)

locally optimal

optimal with additional constraint:

\begin{equation} \norm{z-x}_{2} \leq R \end{equation}

implicit constraints

Of course, all optimization problems have the following constraint implicitly:

\begin{equation}

\end{equation}

why optimization

Headline: instead of saying how to choose some action/model \(x\), you articulate what you want out of the properties of \(x\), then let an algorithm decide on action/model \(x\).

optimization for decision making

“Its a mathematical formulation of making good choices.”

• trades in a portfolio
• airplane controls
• assignment / schedule
• resource allocation

The smaller the objective \(f_{0}\qty(x)\), the better. Constraints limit action space or impose conditions on the outcome. \(x\) represents some kind of actions.

optimization for modeling

Instead of \(x\) representing an action, \(x\) represents the parameters. Constraints impose model parameter requirements; objective \(f_{0}\qty(x)\) is a sum of….

• a prediction error (loss) on observed data
• a (regularization) term that penalizes model complexity

optimization for worst-case analysis

• variables are actions or parameters out of our control
• constraints limit the possible range of parameters
• minimizing \(-f_{0}\qty(x)\) finds worst possible parameter values for your system

optimization-based models

Simulate the dynamics of a system (i.e. what it will do) by giving it the same signals. i.e. model cells by constraints of reactions and optimizing for e.g. growth.

commentary

“Its not interesting to have bigger “margins” to the constraints.”