Houjun Liu

talkbank

Moved much of this to Drafts instead

  • phonbank: poor articulation

  • disfluent kids

  • late talkers

  • Write a review about ASR benchmark methods

    • REV would be our benchmark
      • What corpora we use?
      • Has anyone used disordered speech?
      • Or really seriously accented speech vis a vi CORALL (how was CORALL sampled?)
    • What samples? How do we sample? What are the benchmarks?

  • ASR model + WER

  • tildes and noprompt swapped

  • WER

  • missing words

  • correct alignment

things

  • swap noprompt backwards
  • apostrophies for quotes
  • the word separation error
    • put tilde BETWEEN specific symbols with connection symbols
  • jemoka becomes batchalign 2
  • Extended UD?

combining

  • bash script to run batchalign multiple times throughout the directories

Removing

  • removing non-auditory SBCA corpus area

Diarization

  • Diarization as a Bi-product of ASR

humans at the end

  • do speaker ID in the end
  • DO TO BATCHALIGN
  • allow people to reject files

runhouse meeting

  • <>Donny Greenberg: ADNE, nurses’ health
  • implementation at Google: grantees of canniniminty

Remaining questions

  • but we can’t provide SSH
  • function.save()
  • remote
    • running through hashicorp vault?
    • serializing ssh key remote?
    • RUNHOUSE call into remote!
    • headscale
  • take wave2vec and hubert and GSLM

questions?

  • ask about inter-turn pauses, where
    • INV: something something something <-
    • PAR: WWW <-
    • INV: somethingsomething else <-
    • PAR: words words word
  • no bullets are given for PAR, so do we skip it? do we count the time for WWW all as an inter-turn pause between INV and PAR? etc.

Per Turn

  • Turn level analysis
  • Rename tier to

Silence duration?

  • does it include inter-utterance pauses?

  • within-utterance pause

    • fluency, mechanistic

  • between-utterance pause

    • pause between utterances

  • also: between-speaker pause!

    • leaves room for the speaker to take the floor
    • BETWEEN speaker pauses: “I don’t know what you are asking me”, etc.: “breakdown!”

  • add features: STOPPA, TRESTLE, Wang

https://coryshain.github.io/

featurize

  • saturnino
  • fausa

Questions

  • What features?
  • Where to put them?
  • TalkBankDB
  • How to encode the features?

“How informative are your features”

  • Start coming up with features (TRESTLE, perhaps)
  • Encode them into xarray
  • <> saturnino

stuff

  • make Spanish names list
  • name, city, countries

corpuses

  • SABSAE: santa barbara english
  • CABNC: British english

next

  • ignore any words that goes wrong in the pipeline
  • ~change: noun => n; verb => v, etc.~
  • DET: ignore “DEF”, or perhaps the entir featureless
  • unbulleted VAD exprimentents

errors!

line 1492

*PAR: so ‡ anyway I tiptoe to the front door , open the front door and walk in . •1045194_1050644• %mor: co|so beg|beg adv|anyway pro:sub|I v|+n|tip+n|toe prep|to det:art|the n|front n|door cm|cm adj|open det:art|the n|front n|door coord|and n|walk adv|in . %gra: 1|0|BEG 2|1|BEGP 3|5|JCT 4|5|SUBJ 5|0|ROOT 6|5|JCT 7|9|DET 8|9|MOD 9|6|POBJ 10|5|LP 11|14|MOD 12|14|DET 13|14|MOD 14|5|OBJ 15|14|CONJ 16|15|COORD 17|16|NJCT 18|5|PUNCT

errors?

  • words without features needs to be correctly handled (done in the middle of meeting)
  • 04111 (me ma SOS)
  • nouns shouldn’t mark if it is Com,Neut, should’nt mark if its Com
  • fix PASTP => PAST
  • and does past participles exist?

more

  • Move shua to d(e)
  • Include instructions on how to recreate a broken Conda environment
  • Update the package to conda somehow

move

next steps

  • deal with `n`
  • +… fix
  • remove bullets

results

  • ~ contraction
  • & fused
    • suffix
  • getting rid of punkt in mor
    • , => cm
    • . => no PUNKT, stays

stuff

readme

  • conda init zsh (close shell, open again)
  • .mp4
  • mfa model downloading
  • what’s the difference between online docker install and manual install
  • NLTK Huggingface transformers tokenizers (versining)
  • /opt/homebrew/Caskroom/miniforge/base/envs/aligner/lib/python3.9/site-packages/montreal_forced_aligner/corpus/text_corpus.py; getattr(self, k).update(error_dict[k])

AttributeError: ’list’ object has no attribute ‘update’ FileArgumentNotFoundError: ; line 139

DBA

  • See the data on the frequency of haphax legomina vs. COCA

ESPNet

  • need to talk to Ji Yang

Andrew’s Features

  • Collapse two PAR tiers down
  • Checkpoint per file
  • One corpus prompt per run
  • Handle empty tiers
  • I/P selection crashes! contingency
  • preview the LONGEST segment instead of the top one
  • -i kill in the middle

fixes

  • “my mom’s cryin(g)” [<] mm [l648] (also themmm after)
  • “made her a nice dress” [<] mhm [l1086]
  • “when I was a kid I” &=laughs [l1278]

Others

  • chstring (for uh, mm-hmm)

  • retrace (asr&fa folder)

  • lowcase (caps)

  • rep-join.cut (fixes/)

  • numbers
  • <affirmative>
  • ‘mo data!
    • CallFriend/CallHome (ca-data)
    • ISL?
    • SBCSAE
    • Aphasia + MICASE
    • TBI data
  • Providing a Two-Pass Solution
  • Writing
    • Big description of the pipeline
    • Notion of the pipeline
    • Better tokenization?
  • 8/18
  • Initial segment repetition
  • Extracting studdering
  • Gramatically problematic

mar

  • mar has done a thing and its phoneme level
  • We did it, now automated
  • LEAP data

next actions

  • Aphasia (-apraxia?): classification
  • Child data (EllisWeismer)
  • Dementia

a

  • ~Multiple @Begin/CHECK problem~

  • ~Placement of @Options~

  • ~Strange, missing period~

  • ~Bracket comments should FOLLOW words instead of PRECEEDING them~

  • ~%xwor: line~

  • STICK TO DASHES WHEN DISTRIBUTING BATCHALIGN

  • end the utterance when it ends (incl. inter-utterance pauses)

  • “I” need to be capitalized

  • 11005 (LT)

Align EllisWeismer

Also cool to align:

Alzheimer’s Project

The best way to diagnosing alzhimers’ is from language.

Why this field is needed: to analyze a pre-post test metric.

Desired output: existence of dementia (a.k.a alzheimer’s’).

Other research to read:

python3 ~/mfa_data/batchalign-dist/batchalign.py ~/mfa_data/my_corpus ~/mfa_data/my_corpus_aligned

christan marr paper on MFA on child data