_index.org

Multi-LSTM for Clinical Report Generation

Last edited: August 8, 2025

Take X-Rays and generate clinical reports.

Method

encoder decoder architectures

Encoder

ConViT: convolutional vision transformer. Special thing: we swap out the attention

Double Weighted Multi-Head Attention

We want to force the model to focus on one thing, so we modulate the model based on the weights of other: if one head is big, we make the other head small.

where \(w_{\cos i} = \frac{\sum_{i}^{} \cos \qty (att_{a}, att_{base})}{N}\)

\begin{equation} w = w_{a} \cdot (1- w_{\cos i}) \end{equation}

multiagent reasoning

Last edited: August 8, 2025

simple games

constituents

  • agent \(i \in X\) the set of agents.
  • joint action space: \(A = A’ \times A^{2} \times … \times A^{k}\)
  • joint action would be one per agent \(\vec{a} = (a_{1}, …, a_{k})\)
  • joint reward function \(R(a) = R’(\vec{a}), …, R(\vec{a})\)

additional information

prisoner’s dilemma

CooperateDefect
Cooperate-1, -1-4, 0
Defect0, -4-3, -3

traveler’s dilemma

  • two people write down the price of their luggage, between 2-100
  • the lower amount gets that value plus 2
  • the higher amount gets the lower amount minus 2

joint policy agent utility

for agent number \(i\)

Multimodal AI for Real-World Signals

Last edited: August 8, 2025

Key idea: multi-modality, when leveraged well, leads to faster convergence.

Data Availability

Health and health sensing requires labels, but health signals require specialist knowledge + broader context to label.

  • typical image labeling: 0.05/label
  • medical imaging: 4.00/label

Even if want to automate the study, we need to Kyntic style strap a thing to a person and have soft labels that we align with raw sensor data..

Instead, Do Time-series

Instead: run proxy self-supervised studies into the future—pretraining on a shit tone of sensor data just as timeseries regressing into the future without any labels.

multinomial coefficient

Last edited: August 8, 2025

Its a general form of the combinations formula:

\begin{equation} {n \choose k_1, k_2, \dots, k_{n}} = \frac{n!}{k_{1}! k_2! \dots k_{n}!} \end{equation}