LM Alignment
Last edited: August 8, 2025Been Kim
alignment problem involves “aligning” the representation spaces between machines of the world and that of the human. alternative perspective: teach humans new concepts to understand/communicate better
feature attribution doesn’t work
We take that perspective because many of the intersectional intepretability doesn’t work well (feature permutation, etc.)—feature attribution type analyses (“Impossibility Theorems Been Kim”) actually has no correlation with predictive results.
feature information store in models is unrelated to model edit success
i.e.: knowledge storing location located using ROME technique, though it gives you a sense of the location to store information, doens’t correlate to success of model editing.
Local Policy Search
Last edited: August 8, 2025We begin with a policy parameterized on anything you’d like with random seed weights. Then,
- We sample a local set of parameters, one pertubation \(\pm \alpha\) per direction in the parameter vector (for instance, for a parameter in 4-space, up, down, left, right in latent space), and use those new parameters to seed a policy.
- Check each policy for its utility via monte-carlo policy evaluation
- If any of the adjacent points are better, we move there
- If none of the adjacent points are better, we set \(\alpha = 0.5 \alpha\) (of the up/down/left/right) and try again
We continue until \(\alpha\) drops below some \(\epsilon\).
log laws
Last edited: August 8, 2025\begin{equation} \log a^{b} = b\log a \end{equation}
\begin{equation} \log (ab) = \log a + \log b \end{equation}
\begin{equation} \log (\frac{a}{b}) = \log a - \log b \end{equation}
Logan's Team Checkin
Last edited: August 8, 2025TODO: connect Logan with a few fire departments
logistic equation
Last edited: August 8, 2025Consider:
\begin{equation} P’ = 2P(100-P) \end{equation}
for a motivation, see petri dish.
Solution
Assuming \(P\) never reaches 100
\begin{equation} \int \frac{\dd{P}}{P(100-P)} \dd{P}= \int 2 \dd{t} \end{equation}
Partial fractions time:
\begin{equation} \frac{1}{100} \int \qty(\frac{1}{p} + \frac{1}{100-p})\dd{P} = \frac{1}{100} \ln |p| - \ln |100-p| = 2t+C \end{equation}
Remember now log laws:
\begin{equation} \frac{1}{100} \ln \left| \frac{p}{100-p} \right| = 2t+C \end{equation}
And finally, we obtain:
\begin{equation} \qty | \frac{p}{100-p} | = e^{200t + C} \end{equation}
