• matrix calculus
• supervised learning
• gradient descent
• Newton’s Method
• regression
  • \(y|x,\theta\) is linear Linear Regression
    • least-squares error: probabilistic intuition for least-squares error in linear regression
    • Normal Equation
  • What if \(X\) and \(y\) are not linearly related? Generalized Linear Model
    • \(y|x, \theta\) can be any distribution that’s exponential family
    • some exponential family distributinos: SU-CS229 Distribution Sheet
• classification
  • take linear regression, squish it: logistic regression; for multi-class, use softmax \(p\qty(y=k|x) = \frac{\exp \theta_{k}^{T} x}{\sum_{j}^{} \exp \theta_{j}^{T} x}\)
  • generative learning
    • modeling each class’ distributions, and then check which one is more likely: GDA
    • Naive Bayes
• bias variance tradeoff
• regularization
• unsupervised learning
  • k-means clustering
  • Gaussian mixture model and expectation maximization
    • Jensen’s Inequality
• feature map and precomputing Kernel Trick
• Decision Tree