simple games

constituents

• agent \(i \in X\) the set of agents.
• joint action space: \(A = A’ \times A^{2} \times … \times A^{k}\)
• joint action would be one per agent \(\vec{a} = (a_{1}, …, a_{k})\)
• joint reward function \(R(a) = R’(\vec{a}), …, R(\vec{a})\)

additional information

prisoner’s dilemma

traveler’s dilemma

• two people write down the price of their luggage, between 2-100
• the lower amount gets that value plus 2
• the higher amount gets the lower amount minus 2

joint policy agent utility

for agent number \(i\)

Key idea: multi-modality, when leveraged well, leads to faster convergence.

Data Availability

Health and health sensing requires labels, but health signals require specialist knowledge + broader context to label.

• typical image labeling: 0.05/label
• medical imaging: 4.00/label

Even if want to automate the study, we need to Kyntic style strap a thing to a person and have soft labels that we align with raw sensor data..

Instead, Do Time-series

Instead: run proxy self-supervised studies into the future—pretraining on a shit tone of sensor data just as timeseries regressing into the future without any labels.

Its a general form of the combinations formula:

\begin{equation} {n \choose k_1, k_2, \dots, k_{n}} = \frac{n!}{k_{1}! k_2! \dots k_{n}!} \end{equation}

what if we did Importance Sampling, but…. had multiple proposals?!

notation: \(w_{i}, \tau_{i}\), etc. all correspond to stuff that came from proposal \(q_{i}\).

standard multiple importance sampling (s-MIS)

1. draw samples from current proposals \(\tau_{i} \sim q_{i}\qty(\tau)\)
2. use all of the samples to estimate \(p_{\text{fail}}\)

\begin{equation} \hat{p}_{\text{fail}} = \frac{1}{m} \sum_{i=1}^{m} w_{i} 1\qty {\tau_{i}\not \in \psi} \end{equation}

where

\begin{equation} w_{i} = \qty(\frac{p\qty(\tau_{i})}{q_{i}\qty(\tau_{i})}) \end{equation}

deterministic mixture multiple importance sampling (DM-MIS)

1. draw samples alternating each of the proposals
2. use them to estimate \(p_{\text{fail}}\)

\begin{equation} w_{i} = \frac{p\qty(\tau_{i})}{\frac{1}{m}\sum_{j=1}^{m}q_{j}\qty(\tau_{i})} \end{equation}

\begin{equation} B = \qty[(x_1, y_1), \dots, (x_{n}, y_{n})] \end{equation}

where the labels would be:

\begin{equation} C(b) = \begin{cases} 0, if \sum_{i}^{}y_{i} = 0 \\ 1, \text{otherwise} \end{cases} \end{equation}

and then we maxpool

MILFormer

MILFormer is a multiple-instance learning scheme which makes predictions over input patches whose output predictions are weighted as multi-distirbution.