Memoryless policy search through fake determinism.

• uses a deterministic simulative function to calculate the value
• performs policy search by using normal standard optimizations

Primary contribution: transforming stochastic POMDP to a deterministic simulative function; foregos alpha vectors.

Suppose you have \(m\) initial states that you sampled, you can then just try to get the set of acions that maximize:

\begin{equation} \arg\max_{\theta} \tilde{V} = \frac{1}{m} \sum_{n}^{m} V_{\theta}(s_{m}) \end{equation}

To actually ensure that \(V\) has deterministic transitions…

permits model is a counter for which there is \(n\) threads can do a task. For instance, there is \(n\) permits; each time it is requested, it needs to be subtracted.

Ideally, we do this without busy waiting (while loops with lock and unlocks). So:

condition variable

you can call wait on a condition variable, which will block until another thread calls notify_all.

1. identify a single event to wait/notify
2. ensure that there is something to check to represent the event
3. create a condition variable and share it
4. identify who is the notifier, call notify_all when appropriate
5. identify who will wait, and wait until condition variable triggers

condition_variable_any permitsCV;

// ...

thread(ref(permitsCV))

Identify the ISOLATED event to notify: for instance, whenever permit goes from 0=>1, you notify. But, when permits go from 1=>2, there really isn’t really a need to notify. If you gave wait an unlocked lock, you UB.

permittivity of free space is a constant \(\epsilon_{0} \approx 8.85 \times 10^{-12} \frac{C^{2}}{N \cdot m^{2}}\).

redefinition of Coulomb’s Constant based on permittivity of free space

\begin{equation} k = \frac{1}{4\pi \epsilon_{0}} \end{equation}

A permutation \(\pi\) of some \(\{1,2,…, n\}\) is a rearrangement of this list. There are \(n!\) different permutations of this set.

A permutation is an ORDERED arrangement of objects.

permutation with indistinct objects

What if you want to order a set with sub-set of indistinct objects? Like, for instance, how many ways are there to order:

\begin{equation} 10100 \end{equation}

For every permutation of \(1\) in this set, there are two copies being overcounted.

Suppose you have a series of column pivots:

\begin{equation} \qty(P_{r_2} M_{11} P_{r_1} \dots) \end{equation}

you can combine all row permutations and column permutations down