General Information

> Please indicate which prompt you have selected (Q1, Q2, or Q3) at the beginning of your essay.

In recent years, political polarization has increased in many democratic societies. As Diamond has observed, “among the liberal democracies, partisan and ideological polarization is often worrisomely high, while political tolerance and trust have eroded.” This trend also manifests itself in the growing ideological distance between political parties, increasing partisanship among the electorate, and the erosion of civility in public discourse.

Key Information

• Title: Fine-Grained Language Model Detoxification with Dense, Token-Level Rewards
• Team Member (in 224n): Houjun Liu <[email protected]>
• External Collaborators: Amelia Hardy <[email protected]>, Bernard Lange <[email protected]>
• Custom Project
• Mentor: we have no particular mentor within 224n
• Sharing Project: this project is shared with AA222, and is a part of a research project PI’d by Mykel Kochenderfer <[email protected]>, of which Houjun is taking a leading role

Research Paper Summary

Background

Reinforcement Learning with Human Feedback (RLHF) has demonstrated superb effect for improving performance of a language model (LM) via human preference judgments of LM output desirability–reducing incidences of toxic or false generation trajectories ((Ziegler et al. 2020)). Naive application of RLHF directly has shown success in reducing the toxicity in language model outputs, yet its effects could sometimes be inconsistent without further in-context guidance of the resulting model ((Ouyang et al. 2022)).

Background

Current deep-learning first approaches have shown promising results for the speech text diarization task. For ASR-independent diarization, specifically, two main methods appear as yielding fruitful conclusions:

1. Auditory feature extraction using deep learning to create a trained, fixed-size latent representation via Mel-frequency cepstral coefficients slices that came from any existing voice-activity detection (VAD) scheme ((Snyder et al. 2018)), where the features extracted with the neural network are later used with traditional clustering and Variational Bayes refinement ((Sell et al. 2018; Landini et al. 2022)) approaches to produce groups of diarized speakers

A matrix is upper-triangular if the entries below the diagonal are \(0\):

\begin{equation} \mqty(\lambda_{1} & & * \\ & \ddots & \\ 0 & & \lambda_{n}) \end{equation}

properties of upper-triangular matrix

Suppose \(T \in \mathcal{L}(V)\), and \(v_1 … v_{n}\) is a basis of \(V\). Then:

1. the matrix of \(T\) w.r.t. \(v_1 … v_{n}\) is upper-triangular
2. \(Tv_{j} \in span(v_1 \dots v_{j})\) for each \(v_{j}\)
3. \(span(v_{1}, … v_{j})\) is invariant under \(T\) for each \(v_{j}\)

\(1 \implies 2\)

Recall that our matrix \(A=\mathcal{M}(T)\) is upper-triangular. So, for any \(v_{j}\) sent through \(A\), it will be multiplied to the $j$-th column vector of the matrix. Now, that $j$-th column has \(0\) for rows \(j+1 … n\), meaning that only through a linear combination of the first \(j\) vectors we can construct \(T v_{j}\). Hence, \(Tv_{j} \in span(v_1 … v_{j})\)

Gah I have to do this. Not for public consumption. California laws 2022 DL600 R7 2022.

Consequences

Not licensed

• If unlicensed person is drivnig your car, it maybe impounded for 30 days
• Hired to drive interstate commercially need to be older than 21, also need to be older than 21 to transport hazardous materials

Class C License

Driving #knw

• Two axle vehicle with a GVWL of 26,000 lbs or less
• Three axle vehicle weighing 6,000 lbs or less
• House car < 40 feet or less
• Three wheel motocycles
• Vanpool vehicle designed to carry between 10 and no more than 15 people

Towing #knw

• Single vehicle of 10,000 or less
• Vehicle weighing 4000 lbs or more unladen
  • Trailer coach under 10,000 lbs
  • Fifth wheel trailer exceeding 10,000 lbs but under 15,000 lbs, with endorsement

Mor ethings

• Class C drivers can’t tow more than one
• Motor vehile weigning under 4000 lbs cannot tow more than 6000 lbs

Getting in trouble

• Get a traffic ticket and fail to show up to court: suspend driving
• One at fault collision or one at fault traffic violation: may take action?
• Two of either at fault collision or violation conviction: no driving for 30 days unless accompanied by 25 year old adult
• Three of “”: no driving for 6 months, on probation for a year.
• Drugs or alcohol between 13-21: suspension for a year

Minor driving

Not sure if this applies