DOI: 10.3389/fcomp.2021.624694

One-Liner

Proposed a large multimodal approach to embed auditory info + biomarkers for baseline classification.

Novelty

Developed a massively multimodal audio-to-embedding correlation system that maps audio to biomarker information collected (mood, memory, respiratory) and demonstrated its ability to discriminate cough results for COVID. (they were looking for AD; whoopsies)

Notable Methods

• Developed a feature extraction model for AD detection named Open Voice Brain Model
• Collected a dataset on people coughing and correlated it with biomarkers

Key Figs

Figure 2

This is MULTI-MODAL as heck

see Lambda Calculus

effability

see also language

Transitions

1. Transition first from rule based learning to statistical learning
2. Rise of semantic parsing: statistical models of parsing
3. Then, moving from semantic parsing to large models—putting decision making and language modeling into the same bubble

Importance of LLMs

• They are simply better at understanding language inputs
• They can generate structured information (i.e. not just human language, JSONs, etc.)
• They can perform natural language “reasoning”—not just generate

(and natural language generation, abv)