perplexity is a measure of a language model’s ability to predict words.

Intuition

A good language model should prefer “real” or otherwise “frequently observed” sentences. That is, it should assign lower probability to word salad.

So a good language model should assign a higher probability to the next word that actually occurs given a sequence of words.

Generally, we want the LM to assign high probability to the entire test set. However, big issue is that probability gets smaller by length of the text.

PET is a type of plastic.

Consider a family of bacterial:

\begin{equation} P’ = 2P \end{equation}

this is a normal exponential growth situation. However, we know this isn’t true. Because the nutrients in the petri dish has a finite amount of nutrients. Hopefully this rule succeeds when the population is small, and should stop when the growth is bounded.

For instance, say you can never have more than 100 bacteria:

\begin{equation} P’ = 2P(100-P) \end{equation}

See logistic equation for solution

PGA extends controller gradient ascent to cover CPOMDPs

Notation

Recall from controller gradient ascent we have an objective which we will modify for CPOMDPs. For initial controller-states \(\beta\) and utility \(\bold{u}_{\theta}\):

\begin{equation} \max_{\theta}\ \beta^{\top} (\bold{I} - \gamma \bold{T}_{\theta})^{-1} \bold{r}_{\theta} \end{equation}

subject to:

• \(\Psi\) remains a probably distribution over \(|A|\)
• \(\eta\) remains a probably distribution over \(|X|\)
• and, new for CPOMDP, \(\beta^{\top} (\bold{I} - \gamma \bold{T}_{\theta})^{-1} C_{i} \leq \epsilon_{i}\ \forall i\), that is, each constraint \(C_{i} \in \bold{C}_{i}\) is satisfied to be lower than the budget \(\epsilon_{i}\).

where

\begin{equation} y’ = f(y) \end{equation}

for autonomous ODEs, we can plot a phase line

because autonomouse ODEs, we can plot such a line whereby we can analyze the direction of a solution function’s travel

a particle’s one-way motion must converge to a stationary value, or \(\pm \infty\), as \(t\) increases