Key Sequence

Notation

New Concepts

• worst-case analysis
• Big-oh
• sorting
  • insertion sort
  • merge sort

Important Results / Claims

quantifying success

• single example
• did it on IID examples
• worst-case analysis

Logistics

Driving Forces Behind AI

Three key players

Computation

• cloud compute
• GPUs

Data

Web data is a powerful source to get more general intelligence.

Algorithms

Old school algorithms, but then with enough data scales.

Key Trends in AI

1. moving away from symbolics: don’t code, learn
2. moving away from small networks: deep learning
3. LLMs popularizing machine learning
4. more ethical AI

Scale Emergence

Capabilities in LMs emerge after certain scale: i.e. there’s a sudden improvement in performance after a while.

Supervise learning!

Some Notational Conventions

• \(n\): number of training examples
• \(m\): number of features
• \(x\): input feature(s)
• \(y\): output*/*target feature
• \(\theta\): parameters
• \(h_{\theta}\qty(x)\): the predictor function

And so, a tuple \(\qty(x,y)\) is a particular training example. We will use the parentheses notation to denote samples, so \(\qty(x^{(i)}, y^{(i)})\) as the ith example of training. We typically use \(h\qty(x)\) as the predictor, parameters are \(\theta_{j}\).

New Concepts

• Linear Regression
  • least-squares error
  • gradient descent
    • gradient descent for least-squares error
    • variants
      • summing over dataset: batch gradient descent
      • pick one sample and run it: stochastic gradient descent
      • pick some samples and run them: mini-batch gradient descenmini-bach gradient descet
• a primer on Vector Calculus
  • trace
• Normal Equation

Key Sequence

Review even more! Linear Regression, give some intuition, discuss logistic regression and give an optimization method for it.

Notation

Recall the notation:

• \(\qty(x^{(i)}, y^{(i)})\), ith example
• \(x^{(i)} \in \mathbb{R}^{m+1}\), where \(x_0^{(i)}, \forall i = 1\)
• \(y^{(i)} \in \mathbb{R}\)

\(n\) — number of examples; \(m\) — number of features

New Concepts

• Locally-Weighted Regression
• logistic regression
• Newton’s Method
• parametricity of learning algorithms
  • Non-Parametric Learning Algorithms
  • Parametric Learning Algorithm

Important Results / Claims

• probabilistic intuition for least-squares error

Questions

Interesting Factoids

Scratch

Homo Sapiens are animals that depend on ecosystems. Our social system depends on ecosystems.