EE364A.stanford.edu

Lecture

• SU-EE364a JAN062026
• SU-EE364A JAN082026
• SU-EE364A JAN132026
• SU-EE364A JAN152026
• SU-EE364A JAN202026
• SU-EE364A JAN222026
• SU-EE364A JAN272026
• SU-EE364A FEB052026
• SU-EE364A FEb102026

motivation

Consider Generic Maximum Likelihood Estimate.

• parametric distribution estimation: suppose you have a family of densities \(p_{x}\qty(y)\), with parameter \(x\)
• we take \(p_{x}\qty(y) = 0\) for invalid values of \(x\)

maximum likelihood estimation: choose \(x\) to maximize \(p_{x}\qty(y)\) given some dataset \(y\).

linear measurement with IID noise

Suppose you have some kind of linear noise model:

\begin{equation} y_{i} = a_{i}^{T}x + v_{i} \end{equation}

where \(v_{i}\) is IID noise, and \(a^{T}_{i}\) is the model. We can write \(y\) probabilistically as:

Consider an error minimization task \(\min \norm{Ax - b}\) (\(Ax\) as the “predictions”, and \(b\) is the “data”). Some interpretation—

• approximation: \(Ax^{*}\) is the best approximation of the vector \(b\) by linear combinations of columns of \(A\)
• geometric: \(Ax^{*}\) is a point in \(\mathcal{R}\qty(A)\) closest to \(b\)
• estimation: linear measurement model \(y = Ax + v\)
  • you took a measurement \(y\), \(A\) is the theoretical measurement, \(v\) is the measurement error
  • implausibility of making \(v\) error is \(\norm{v}\)
  • given \(y = b\) (what you measured), most plausible \(x\) is \(x^{*}\)
• optimal design: \(x\) are design variables, \(Ax\) is the result; \(x^{*}\) is the design that best approximates desired \(b\)

Penalty Function Approximation

Suppose you are optimizing some design \(x\) with respect to some dynamics \(A\). Suppose your design residual is \(r = Ax - b\). And you have some kind of penalty function to describe how comfortable you are with various errors: \(\phi\qty(r_1) + … + \phi\qty(r_{n})\).

Let the unpreturb problem be:

\begin{align} \min_{x}\quad & f_{0}\qty(x) \\ \textrm{s.t.} \quad & f_{i}\qty(x) \leq 0, i = 1 \dots m \\ & h_{i}\qty(x) = 0, i = 1 \dots p \end{align}

Preturbed on is just:

\begin{align} \min_{x}\quad & f_{0}\qty(x) \\ \textrm{s.t.} \quad & f_{i}\qty(x) \leq u_{i} \\ & h_{i}\qty(x) = v_{i} \end{align}

Global Tightness

So we can get a lower bound:

\begin{equation} p^{*}\qty(u,v) \geq g\qty(\lambda^{*}, v^{*}) - u^{T} \lambda^{*} - v^{T}\lambda^{*} \end{equation}

Consider the Robust Approximation problem:

Robust Approximation

Minimize \(\norm{Ax - b}\) with uncertain \(A\); two approaches—

1. stochastic: minimize \(\mathbb{E}\norm{Ax -b }\)
2. worst-case: set \(\mathcal{A}\) of possible values of \(A\), minimize \(\text{sup}_{A \in A} \norm{Ax - b}\)