_index.org

SU-CS224N MAY022024

Last edited: October 10, 2026

Zero-Shot Learning

GPT-2 is able to do many tasks with not examples + no gradient updates.

Instruction Fine-Tuning

Language models, by default, are not aligned with user intent.

  1. collect paired examples of instruction + output across many tasks
  2. then, evaluate on unseen tasks

~3 million examples << n billion examples

dataset: MMLU

You can generate an Instruction Fine-Tuning dataset by asking a larger model for it (see Alpaca).

Pros + Cons

  • simple and straightforward + generalize to unseen tasks
  • but, its EXPENSIVE to collect ground truth data
    • ground truths maybe wrong
    • creative tasks may not have a correct answer
    • LMs penalizes all token-level mistakes equally, but some mistakes are worse than others
    • humans may generate suboptimal answers

Human Preference Modeling

Imagine if we have some input \(x\), and two output trajectories, \(y_{1}\) and \(y_{2}\).

SU-CS329H OCT192026

Last edited: October 10, 2026

Key Sequence

Notation

New Concepts

Important Results / Claims

why not prompting

Even if its fast, no training needed, but its still not good for complex / nuanced behaviors. Models stop following long prompts, and rewards models are very very specific.

“Articulability”

Questions

Interesting Factoids

Baysian Estimation for Preference Models

Last edited: October 10, 2026

Consider the case where we have \(y_{t}=1\) where \(C\) wins, and \(y_{t}=0\) where \(D\) wins. Recall that our model is:

\begin{equation} D \sim \text{Bern}\qty(\sigma\qty(\delta)) \end{equation}

where some \(\delta = s_{C} - s_{D}\). Now, let’s be Bayesian and compute:

\begin{equation} \delta \sim \mathcal{N}\qty(0, \tau^{2}) \end{equation}

where we say \(\delta\) starts at some measure \(\tau\).

MAP estimate

We want to find the mode of the posterior. So, we can write a MAP given this setup with Bayes rule:

DAC Fun

Last edited: October 10, 2026

Fun with playing around the TI 5121 DAC.

modes

pp 6:

i2c interface

pp 55:

Write: 0x98 - 10011000 Read: 0x99 - 10011001

i2s interface on BCM

page 119

but how do I set the master clock?

Heh. Its not documented. Go figure. Fortunately, https://firmanhp.com/assets/20241005-id-raspi-bare-metal-rust/pdf/BCM2837-ARM-Peripherals-Revised-V2-1.pdf seems to tell us what to do:

There are two PCM/I2S clock control registers.

CM_PCM_CTRL @ 0x7E101098

  • Bit 5 = Emergency Stop; should be 0; debugging only
  • Bit 4 = Clock Enable
  • Bits 3:0 = Clock Source

OSC (SRC=0b0001) is cool and allegedly good, but it runs on 19.2 which is less than crispy crispy 48khz so we are going to use PLLD (SRC=0b0110) which runs at a cheeky 500MHz nominal.

Houjun's Academic Home Page

Last edited: October 10, 2026

👋 Howdy, I'm Houjun Liu!

I’m a third-year coterminal MSCS and BSCS student in the Computer Science Department at Stanford University, grateful to be advised by Prof. Mykel Kochenderfer. In the course of my research, I have also had the fortunate opportunity to work with Stanford NLP under Prof. Chris Manning and Prof. Diyi Yang, CMU TalkBank under Prof. Brian MacWhinney, and Prof. Xin Liu at UC Davis Engineering. I am affiliated with the Stanford NLP Group and Stanford Intelligent Systems Lab. I previously visited Microsoft Research Frontiers as a research intern.