Richard Nixon bombs Vietnam for 13 days to beat the VietCong into submission after the Vietnam War.

A Linear Map from a vector space to itself is called an operator.

\(\mathcal{L}(V) = \mathcal{L}(V,V)\), which is the set of all operators on \(V\).

constituents

• a vector space \(V\)
• a Linear Map \(T \in \mathcal{L}(V,V)\)

requirements

• \(T\) is, by the constraints above, an operator

additional information

injectivity is surjectivity in finite-dimensional operators

Suppose \(V\) is finite-dimensional and \(T \in \mathcal{L}(V)\), then, the following statements are equivalent:

1. \(T\) is invertable
2. \(T\) is injective
3. \(T\) is surjective

THIS IS NOT TRUE IN infinite-demensional vector space OPERATORS! (for instance, backwards shift in \(\mathbb{F}^{\infty}\) is surjective but not injective.)

An opsin is a photo-receptor protein (sensitive to light) that is sensitive to light

Suppose we have offline statistic regarding wins and losses of each slot machine as our state:

\begin{equation} w_1, l_{1}, \dots, w_{n}, l_{n} \end{equation}

What if we want to create a policy that maximises exploration?

We construct a value function:

\begin{equation} U^{*}([w_1, l_{1}, \dots, w_{n}, l_{n}]) = \max_{a} Q^{*}([w_1, l_{1}, \dots, w_{n}, l_{n}], a) \end{equation}

our policy is the greedy policy:

\begin{equation} U^{*}([w_1, l_{1}, \dots, w_{n}, l_{n}]) = \arg\max_{a} Q^{*}([w_1, l_{1}, \dots, w_{n}, l_{n}], a) \end{equation}

1. Shuffle cards
2. Keep revealing cards
3. “Stop” when there’s >50% chance the next card to be revealed is black

We can Frequentist Definition of Probability calculate the probability of a given card remaining is black:

\begin{equation} pblack(b,r) = \frac{26-b}{52-(r+b)} \end{equation}

now:

\begin{equation} pwin(b,r) = \begin{cases} 0, b+r = 52 \\ \max \qty[ \begin{align}&pblack(p,r), \\ &pblack(b,r)pwin(b+1,r) + (1-pblack(b,r)pwin(b, r+1) \end{align}] \end{cases} \end{equation}

“with the theory of the Martingales, this comes out to be 50%”