You can collect feedback depending on which model you’d like!

• Are you facing a PAIR of items’s rank?: Bradley-Terry Preference Model.
• Are you facing ONE item’s acceptability?: Rasch Model (additive model)

itemwise feedback

• Hey dude, do you prefer \(C\)?
• Hey dude, do you prefer \(D\)?

pairwise feedback

Ground truth collected as: “hey dude, which one do you prefer? \(C\) or \(D\)?”

additional information

Rasch Model cannot be applied pairwise feedback

Suppose you really liked itemized feedback but then you want to extract a pairwise preferences. That means, you collected itemiwise data, and learned some model \(r\qty(i,j) = \theta_{i} + b_{j}\) following Rasch Model, which you normally then take a sigmoid and ask whether \(P\qty(y_{ij} = 1) = \sigma\qty(r_{ij})\).

But then you got hit on the head and want to move to a Bradley-Terry Preference Model. You cannot recycle this model. Because:

\begin{align} P\qty(a \succ b \mid i) &=\sigma\qty(r_{ia} - r_{ib}) \\ &= \sigma \qty(\theta_{a} + b_{i} - \theta_{b} - b_{i}) \\ &= \sigma \qty(\theta_{a} - \theta_{b}) \end{align}

and notice that under this basic linear model your learned preference data cannot represent heterogeneity between users (??? but i and j would be different?).

how to solve? use factor model!