Some more improvements to Importance Sampling.

Cross Entropy Method

1. draw initial samples
2. fit a new distribution with the subset that failed: weight each sample by

\begin{equation} w\qty(\tau) = \frac{p\qty(\tau) \qty {\tau \not \in \psi}}{q\qty(\tau)} \end{equation}

problem: what if, immediately on the first proposal, we never got any failures? Then the weight of everything is zero and then life is bad.

adaptive cross entropy method with adaptive importance sampling

Pick a notion of “distance to failure” \(f\qty(\tau)\)

Operation that adds elements in a set

constituents

• A set \(V\)
• Each non-necessarily-distinct elements \(u,v \in V\)

requirements

addition on a set \(V\) is defined by a function that assigned an element named \(u+v \in V\) (its closed), \(\forall u,v\in V\)

additional information

See also addition in \(\mathbb{F}^n\)

The additive identity allows another number to retain its identity after adding. That is: there exists an element \(0\) such that \(v+0=v\) for whatever structure \(v\) and addition \(+\) you are working with.

Assume for the sake of contradiction \(\exists\ 0, 0’\) both being additive identities in vector space \(V\).

Therefore:

\begin{equation} 0+0’ = 0’ +0 \end{equation}

commutativity.

Therefore:

\begin{equation} 0+0’ = 0 = 0’+0 = 0' \end{equation}

defn. of identity.

Hence: \(0=0’\), \(\blacksquare\).

Take a vector \(v \in V\) and additive inverses \(a,b \in V\).

\begin{equation} a+0 = a \end{equation}

defn. of additive identity

\begin{equation} a+(v+b) = a \end{equation}

defn. of additive inverse

\begin{equation} (a+v)+b = a \end{equation}

associativity

\begin{equation} 0+b = a \end{equation}

defn. of additive inverse

\begin{equation} b=a\ \blacksquare \end{equation}