Proposed a large multimodal approach to embed auditory info + biomarkers for baseline classification.
Developed a massively multimodal audio-to-embedding correlation system that maps audio to biomarker information collected (mood, memory, respiratory) and demonstrated its ability to discriminate cough results for COVID. (they were looking for AD; whoopsies)
- Developed a feature extraction model for AD detection named Open Voice Brain Model
- Collected a dataset on people coughing and correlated it with biomarkers
This is MULTI-MODAL as heck
This figure tells us the large network the came up with.
Table 2 and 3
The descriminator tacked on the end of the network is transfer-trained to different tasks. It shows promising results for cough-to-COVID classification
Is biomarker data something that is commonly used as a feature extraction/benchmark tool?