Three shapes of how humans express preferences:
accept-reject
constituents
Rater \(i\) sees \(j\), and gives \(y_{ij} \in \qty {1,0}\). (thumbs up or down)
requirements
We can write down this as the following model:
\begin{equation} P\qty(y_{ij} = 1) = \sigma \qty(\theta_{i} - b_{j}) \end{equation}
where \(\theta_{i}\) is some notion, and \(b_{j}\) is how “hard” \(j\) is to like, and \(\sigma\) is sigmoid.
Alternatively you can write \(b_{j}\) as appeal and make it additive.
additional information
“test model”
The above model makes an independence assumption, sorta like the .
generalization
This already allow us to learn \(\theta\) and \(b\) separately.
ranking
net promoter score
latent score \(\theta_{i} - b_{j} + \epsilon\), and cut points \(c_1 < c_2 < c_3< c_4\), such that
\begin{equation} P\qty(Y \leq k) = F\qty(c_{k} - \theta_{i} + b_{j}) \end{equation}
choices form sets
Rater \(i\) picks from menu \(T = \qty {j,j’, \dots}\), such that \(C_{i}\qty(T) = j\) meaning element \(j\) was chosen from \(T\).
Write \(j \succ_{i} j’\) if:
\begin{equation} C_{i}\qty(\qty {j,j’}) = j \end{equation}
where \(C\) is a choice function of agent \(i\). We often ask questions about say \(P\qty(j \succ_{i} j’)\) and figt models for it.
two types of inferences
- known: infer \(C_{i} \qty(T)\) when we have observation on \(i\) and \(T\)
- unknown: infer \(C_{i} \qty(T)\) when we have observation on \(i\) and some under “bundle” \(T’\)
axioms of set choices
- infinite data: that is, claims of probability is frequentist exact
- no ties, so \(P\qty(j \succ j’) = 1- P\qty(j \succ j’)\)
-\(P\qty(j \succ j’) > 0\)
pop quiz
Question: \(P\qty(j \succ j’) = P\qty(j’ \succ j’’) = \frac{1}{2}\), when what is \(P\qty(j \succ j’’)\)?
IDK! Anything. You can take two such independetn populations and mix them arbitrarily to get any \(P\qty(j \succ j’’)\) value.
two models of choices of sets
You can either consider such choices are numerical comparisons between choices “scale”"Bradley-Terry Preference Model" view; or you can consider them as “population”"softmax model of choice from sets".
Bradley-Terry Preference Model
The philosophy that choices are “a scale”
See Bradley-Terry Preference Model
softmax model of choice from sets
The philosophy that choices are “a mixture”
All three statements are equivalent.
- Logits: There exists \(u : X \to \mathbb{R}\) such that: \(P\qty(C\qty(S) = j) = \text{softmax}_{j} \qty(u\qty(j’)_{j’ \in S})\)
- independence to irrelavent alternatives: for \(j, j’\) in both \(S\) and \(T\), we have: \(\frac{P\qty(C\qty(S) = j)}{P\qty(C\qty(S) = j’)} = \frac{P\qty(C\qty(T) = j)}{P\qty(C\qty(T) = j’)}\)
- Chain Rule: \(P\qty [C\qty(S) = j] = P\qty [C\qty(T) = j] \qty(\sum_{j’ \in T}^{} P\qty[C\qty(S) = j’])\) “the probability of \(j\) being chosen in \(S\) is the probabliity of \(j\) being chosen in \(T\) times the probability of anybody in \(S\) being chosen”
(1 => 3)
Proof idea: just expand the chain rule by cancelling out softmax denominator:
Write \(P_{s}\qty(j)\) to mean \(P\qty(C\qty(S) = j)\).
Define \(Z_{A} = \sum_{k \in A}^{} e^{u_{k}}\) the bottom of softmax, then for any \(j \in T \subseteq S\), we have
\begin{equation} P_{T}\qty(j)\sum_{j \in T}^{} P_{s}\qty(j) = \frac{e^{u_{j}}}{Z_{T}} \sum_{j \in T}^{} \frac{e^{u_{j}}}{Z_{S}} = \frac{e^{u_{j}}}{Z_{S}} = P_{S}\qty(j) \end{equation}
(3 => 2)
\(j, j’ \in S,T\), define the “overlap” \(R = S \cup T\), then chain rule gives:
\begin{equation} \frac{P_{s}\qty(j)}{P_{s}\qty(j’)} = \frac{P_{R}\qty(j)}{P_{R}\qty(j’)} \frac{\sum_{k \in R}^{} P_{S}\qty(k)}{\sum_{k \in R}^{} P_{T}\qty(k)} = \frac{P_{R}\qty(j)}{P_{R}\qty(j’)} = \frac{P_{T}\qty(j)}{P_{T}\qty(j’)} \end{equation}
where the last equality is applying the chain rule one again in a similar manner.
Ok, so this is great, but suppose your choice function is restricted to a pair e.g., \(C\qty(\qty{j, j’})\) the above is hard to talk about because then the sets \(S\) and \(T\) (maybe disjoint? either way it desn’t typecheck)
So we turn to…