SU-CS224N MAY022024
Last edited: October 10, 2026Zero-Shot Learning
GPT-2 is able to do many tasks with not examples + no gradient updates.
Instruction Fine-Tuning
Language models, by default, are not aligned with user intent.
- collect paired examples of instruction + output across many tasks
- then, evaluate on unseen tasks
~3 million examples << n billion examples
dataset: MMLU
You can generate an Instruction Fine-Tuning dataset by asking a larger model for it (see Alpaca).
Pros + Cons
- simple and straightforward + generalize to unseen tasks
- but, its EXPENSIVE to collect ground truth data
- ground truths maybe wrong
- creative tasks may not have a correct answer
- LMs penalizes all token-level mistakes equally, but some mistakes are worse than others
- humans may generate suboptimal answers
Human Preference Modeling
Imagine if we have some input \(x\), and two output trajectories, \(y_{1}\) and \(y_{2}\).
SU-CS329H OCT192026
Last edited: October 10, 2026Key Sequence
Notation
New Concepts
Important Results / Claims
why not prompting
Even if its fast, no training needed, but its still not good for complex / nuanced behaviors. Models stop following long prompts, and rewards models are very very specific.
“Articulability”
Questions
Interesting Factoids
Baysian Estimation for Preference Models
Last edited: October 10, 2026Consider the case where we have \(y_{t}=1\) where \(C\) wins, and \(y_{t}=0\) where \(D\) wins. Recall that our model is:
\begin{equation} D \sim \text{Bern}\qty(\sigma\qty(\delta)) \end{equation}
where some \(\delta = s_{C} - s_{D}\). Now, let’s be Bayesian and compute:
\begin{equation} \delta \sim \mathcal{N}\qty(0, \tau^{2}) \end{equation}
where we say \(\delta\) starts at some measure \(\tau\).
MAP estimate
We want to find the mode of the posterior. So, we can write a MAP given this setup with Bayes rule:
DAC Fun
Last edited: October 10, 2026Fun with playing around the TI 5121 DAC.
modes
pp 6:

i2c interface
pp 55:

Write: 0x98 - 10011000 Read: 0x99 - 10011001
i2s interface on BCM
page 119

but how do I set the master clock?
Heh. Its not documented. Go figure. Fortunately, https://firmanhp.com/assets/20241005-id-raspi-bare-metal-rust/pdf/BCM2837-ARM-Peripherals-Revised-V2-1.pdf seems to tell us what to do:
There are two PCM/I2S clock control registers.
CM_PCM_CTRL @ 0x7E101098
- Bit 5 = Emergency Stop; should be 0; debugging only
- Bit 4 = Clock Enable
- Bits 3:0 = Clock Source
OSC (SRC=0b0001) is cool and allegedly good, but it runs on 19.2 which is less than crispy crispy 48khz so we are going to use PLLD (SRC=0b0110) which runs at a cheeky 500MHz nominal.
Houjun's Academic Home Page
Last edited: October 10, 2026👋 Howdy, I'm Houjun Liu!
I’m a third-year coterminal MSCS and BSCS student in the Computer Science Department at Stanford University, grateful to be advised by Prof. Mykel Kochenderfer. In the course of my research, I have also had the fortunate opportunity to work with Stanford NLP under Prof. Chris Manning and Prof. Diyi Yang, CMU TalkBank under Prof. Brian MacWhinney, and Prof. Xin Liu at UC Davis Engineering. I am affiliated with the Stanford NLP Group and Stanford Intelligent Systems Lab. I previously visited Microsoft Research Frontiers as a research intern.
