constraint

recall constraint; our general constraints means that we can select \(f\) within a feasible set \(x \in \mathcal{X}\).

active constraint

an “active constraint” is a constraint which, upon application, changes the solution to be different than the non-constrainted solution. This is always true at the equality constraint, and not necessarily with inequality constraints.

types of constraints

We can write all types of optimization problems into two types of constraints; we will use these conventions EXACTLY:

Key Sequence

Notation

• Stochastic Robust Least Squares
  • worst-case robust least-squares
• Maximum Likelihood Estimation with Convex Optimization

New Concepts

• change of variables

Important Results / Claims

Questions

Interesting Factoids

• fun fact: any (even non-convex) problem in two quadratics usually has zero duality gap and thus its dual can be the solution to the primal problem

Consider some kind of ellipsoid where your data is constrained:

\begin{align} \mathcal{A} = \qty {\bar{A} + u_1 A_1 + \dots + u_{p} A_{p} \mid \norm{u}_{2} \leq 1} \end{align}

You can form the “worst-case robust least squares”:

\begin{align} \min \text{sup}_{A \in \mathcal{A}} \norm{A x - b}_{2}^{2} = \min \text{sup}_{\norm{u}_{2} \leq } \norm{P\qty(x) u + q\qty(x)}_{2}^{2} \end{align}

This is usually a minimax problem, but taking the dual of the inner maximize thing turns out has zero duality gap.

• norm approximation