In the event your domain knowledge can help you make decisions about how spark load-balances or stripes data across worker nodes.

Persistence

“you should store this data in faster/slower memory”

MEMORY_ONLY, MEMORY_ONLY_SER, MEMORY_AND_DISK, MEMORY_AND_DISK_SER, DISK_ONLY

rdd.persist(StorageLevel.MEMORY_AND_DISK)
# ... do work ...
rdd.unpersist()

Parallel Programming

options are derivatives which gives you the permission to make a transaction at a particular date.

There are two main types of options:

• call: gives permission to buy a security on or before the “exercise” date
• puts: gives permission to sell a security on or before the “exercise” date

For this article, we will define \(S_{t}\) to be the stock price at the time \(t\), \(K\) as the option’s strike price, \(C_{t}\) to be the price of the “call” option, and \(P_{t}\) to be the price of the “put” option at strike price \(K\); lastly \(T\) we define as the maturity date.

an Option (MDP) represents a high level collection of actions. Big Picture: abstract away your big policy into \(n\) small policies, and value-iterate over expected values of the big policies.

Markov Option

A Markov Option is given by a triple \((I, \pi, \beta)\)

• \(I \subset S\), the states from which the option maybe started
• \(S \times A\), the MDP during that option
• \(\beta(s)\), the probability of the option terminating at state \(s\)

one-step options

You can develop one-shot options, which terminates immediate after one action with underlying probability

Optogenetics are a process of neurology circuit investigations:

1. every neuron which expresses as specific change becomes sensitive to light
2. therefore, you can shine a light on the mouse’s brain to control it

This uses a set of molecules named opsins.

Recall: Oracle Turing Machines is a machine which has oracle \(B \subseteq \Gamma^{*}\) which lets you ask “if \(z \in B\), then do something, otherwise do something else” and check that in ONE STEP.

We have:

\begin{equation} P^{B} = \qty {L \mid L \text{{ can be decided by a polynomial-time TM with oracle for $B$ }}} \end{equation}

For example \(P^{\text{SAT}}\) are languages we can decide in polynomial time once we have an oracle in SAT, and \(P^{\text{NP}}\) are languages decidable by some Oracle Polynomial Time \(TM\) for some \(B \in NP\).