Houjun Liu

Laguarta 2021

# ntj

DOI: 10.3389/fcomp.2021.624694


Proposed a large multimodal approach to embed auditory info + biomarkers for baseline classification.


Developed a massively multimodal audio-to-embedding correlation system that maps audio to biomarker information collected (mood, memory, respiratory) and demonstrated its ability to discriminate cough results for COVID. (they were looking for AD; whoopsies)

Notable Methods

  • Developed a feature extraction model for AD detection named Open Voice Brain Model
  • Collected a dataset on people coughing and correlated it with biomarkers

Key Figs

Figure 2

This is MULTI-MODAL as heck

This figure tells us the large network the came up with.

Table 2 and 3

The descriminator tacked on the end of the network is transfer-trained to different tasks. It shows promising results for cough-to-COVID classification

New Concepts


Biomarker correlation

Is biomarker data something that is commonly used as a feature extraction/benchmark tool?