Random thoughts as I scan through the book:

Central framing: learning as a means to generalize + predict

Key Tasks

• regression (predict a value)
• binary classification (sort an example into a boolean class of Y/N)
• multi-class classification (sort an example into multiple classes)
• ranking (sorting an object into relevant order)

Optimality of Bayes Optimal Classifier

If you have the underlying data distribution, we can classify \(y\) from \(x\) by choosing:

Key Sequence

Notation

New Concepts

• Kernel Trick

Important Results / Claims

Questions

Interesting Factoids

Key Sequence

Notation

New Concepts

• Decision Tree

Important Results / Claims

Questions

Interesting Factoids

constituents

• dataset \(\qty {x^{(1)}, \dots, x^{(n)}}\)
• number of clusters \(k\)

requirements

Initialize cluster centroids \(\mu_{1}, …, \mu_{k}\) randomly, and repeat:

• assigning points \(x^{(i)}\) to cluster centers \(c^{(i)}\): for each \(i \in [1, \dots N]\), write \(c^{(i)} = \arg\min_{j} \norm{x^{(i)}- \mu_{j}}^{2}_{2}\)
• update centroids: for each \(j \in [1 \dots k]\), write \(\mu_{j} = \frac{\sum_{i=1}^{n} \mathbbm{1}\qty {c^{(i)}=j} x^{(i)}}{\sum_{i=1}^{n} \mathbbm{1}\qty {c^{(i)}=j}}\)

additional information

some ways to pick better centroid

• sample the data points as the initial centroids
• k-means++

k-means++

• pick uniform data point to be first centroid
• pick next centroid w.r.t. probability proportional to distance to the previous centroid squared

distortion function

Consider the following function: