MCVI solves POMDPs with continuous state space, but with discrete observation and action spaces. It does this by formulating a POMDP as a graph.

Fast algorithms require discretized state spaces, which makes the problem much more difficult to model. MCVI makes continuous representations possible for complex domains.

MC Backup

Normal POMDP Bellman Backup isn’t going to work well with continuous state spaces.

Therefore, we reformulate our value backup as:

\begin{equation} V_{t+1}(b) = \max_{a \in A} \qty(\int_{s} R(s,a)b(s) \dd{s}) + \gamma \sum_{o \in O}^{} p(o|b,a) V_{t}(update(b,a,o)) \end{equation}

• at each point a relevant result is returned, calculate precision
• and then average that
• and then average the precision over all queries

precision

\begin{equation} \frac{tp}{tp + fp} \end{equation}

recall

\begin{equation} \frac{tp}{tp+fn} \end{equation}

accuracy

\begin{equation} \frac{tp + tn}{tp+tn+fp+fn} \end{equation}

f1

\begin{equation} F_1 = \frac{2 (P\cdot R)}{P+R} \end{equation}