lambda calculus
Last edited: August 8, 2025see Lambda Calculus
Lambek Calculus
Last edited: August 8, 2025Language Agents with Karthik
Last edited: August 8, 2025Transitions
- Transition first from rule based learning to statistical learning
- Rise of semantic parsing: statistical models of parsing
- Then, moving from semantic parsing to large models—putting decision making and language modeling into the same bubble
Importance of LLMs
- They are simply better at understanding language inputs
- They can generate structured information (i.e. not just human language, JSONs, etc.)
- They can perform natural language “reasoning”—not just generate
(and natural language generation, abv)
Language Information Index
Last edited: August 8, 2025What makes language modeling hard: resolving ambiguity is hard.
“the chef made her duck”
Contents
Basic Text Processing
- regex
- ELIZA
- tokenization and corpus
- text normalization
- tokenization + Subword Tokenization
- Word Normalization
- lemmatization through morphological parsing
- only take stems from morphemes: porter stemmer
- sentence segmentation
- N-Grams
Edit Distance
DP costs \(O(nm)\), backtrace costs \(O(n+m)\).
Ngrams
Text Classification
Logistic Regression
- Generative Classifier vs Discriminate Classifier
- Logistic Regression Text Classification
- cross entropy loss
- stochastic gradient descent