_index.org

SU-CS229 SEP222025

Last edited: September 9, 2025

Logistics

Driving Forces Behind AI

Three key players

Computation

  • cloud compute
  • GPUs

Data

Web data is a powerful source to get more general intelligence.

Algorithms

Old school algorithms, but then with enough data scales.

  1. moving away from symbolics: don’t code, learn
  2. moving away from small networks: deep learning
  3. LLMs popularizing machine learning
  4. more ethical AI

Scale Emergence

Capabilities in LMs emerge after certain scale: i.e. there’s a sudden improvement in performance after a while.

SU-CS229 SEP242025

Last edited: September 9, 2025

Supervise learning!

Some Notational Conventions

  • \(n\): number of training examples
  • \(m\): number of features
  • \(x\): input feature(s)
  • \(y\): output*/*target feature
  • \(\theta\): parameters
  • \(h_{\theta}\qty(x)\): the predictor function

And so, a tuple \(\qty(x,y)\) is a particular training example. We will use the parentheses notation to denote samples, so \(\qty(x^{(i)}, y^{(i)})\) as the ith example of training. We typically use \(h\qty(x)\) as the predictor, parameters are \(\theta_{j}\).

New Concepts

SU-CS229 SEP292025

Last edited: September 9, 2025

Key Sequence

Review even more! Linear Regression, give some intuition, discuss logistic regression and give an optimization method for it.

Notation

Recall the notation:

  • \(\qty(x^{(i)}, y^{(i)})\), ith example
  • \(x^{(i)} \in \mathbb{R}^{m+1}\), where \(x_0^{(i)}, \forall i = 1\)
  • \(y^{(i)} \in \mathbb{R}\)

\(n\) — number of examples; \(m\) — number of features

New Concepts

Important Results / Claims

Questions

Interesting Factoids

Scratch

SU-MED103 SEP252025

Last edited: September 9, 2025

Homo Sapiens are animals that depend on ecosystems. Our social system depends on ecosystems.

Vector Calculus Primer

Last edited: September 9, 2025

A primer on Vector Calculus.

trace

constituents

for square \(A \in \mathbb{R}^{m\times m}\), we write:

requirements

\(\text{tr}\qty(A) = \sum_{i}^{} A_{ii}\) is the sum of the diagonals

additional information

properties of traces

\begin{equation} \text{tr}\qty(AB) = \text{tr}\qty(BA) \end{equation}

\begin{equation} \text{tr}\qty(ABC) = \text{tr}\qty(CAB) \end{equation}

\begin{equation} \nabla_{A} \qty [\text{tr}\qty(AB)] = B^{T} \end{equation}