permittivity of free space is a constant \(\epsilon_{0} \approx 8.85 \times 10^{-12} \frac{C^{2}}{N \cdot m^{2}}\).

redefinition of Coulomb’s Constant based on permittivity of free space

\begin{equation} k = \frac{1}{4\pi \epsilon_{0}} \end{equation}

A permutation \(\pi\) of some \(\{1,2,…, n\}\) is a rearrangement of this list. There are \(n!\) different permutations of this set.

A permutation is an ORDERED arrangement of objects.

permutation with indistinct objects

What if you want to order a set with sub-set of indistinct objects? Like, for instance, how many ways are there to order:

\begin{equation} 10100 \end{equation}

For every permutation of \(1\) in this set, there are two copies being overcounted.

Suppose you have a series of column pivots:

\begin{equation} \qty(P_{r_2} M_{11} P_{r_1} \dots) \end{equation}

you can combine all row permutations and column permutations down

perplexity is a measure of a language model’s ability to predict words.

Intuition

A good language model should prefer “real” or otherwise “frequently observed” sentences. That is, it should assign lower probability to word salad.

So a good language model should assign a higher probability to the next word that actually occurs given a sequence of words.

Generally, we want the LM to assign high probability to the entire test set. However, big issue is that probability gets smaller by length of the text.

PET is a type of plastic.