constituents

• where \(x \in \mathbb{R}^{n}\) is a vector of variables
• \(f_{0}\) is the objective function, “soft” to be minimized
• \(f_{1} … f_{m}\) are the inequality constraints
• \(g_{1} … g_{p}\) are the inequality constraints

requirements

Generally of structure:

\begin{equation} \min f_{0}\qty(x) \end{equation}

subject to:

\begin{align} f_{i} \qty(x) \leq 0, i = 1 \dots m \\ g_{i}\qty(x) = 0, i = 1 \dots p \end{align}

solving optimization problems

You can’t generally solve optimization problems… Some types

• convex optimization problems, you can solve these
• non-linear optimization

additional information

why optimization

Headline: instead of saying how to choose some action/model \(x\), you articulate what you want out of the properties of \(x\), then let an algorithm decide on action/model \(x\).

optimization for decision making

“Its a mathematical formulation of making good choices.”

• trades in a portfolio
• airplane controls
• assignment / schedule
• resource allocation

The smaller the objective \(f_{0}\qty(x)\), the better. Constraints limit action space or impose conditions on the outcome. \(x\) represents some kind of actions.

optimization for modeling

Instead of \(x\) representing an action, \(x\) represents the parameters. Constraints impose model parameter requirements; objective \(f_{0}\qty(x)\) is a sum of….

• a prediction error (loss) on observed data
• a (regularization) term that penalizes model complexity

optimization for worst-case analysis

• variables are actions or parameters out of our control
• constraints limit the possible range of parameters
• minimizing \(-f_{0}\qty(x)\) finds worst possible parameter values for your system

optimization-based models

Simulate the dynamics of a system (i.e. what it will do) by giving it the same signals. i.e. model cells by constraints of reactions and optimizing for e.g. growth.

commentary

“Its not interesting to have bigger “margins” to the constraints.”