Generalized thurstone models for ranking: Equivalence and reversibility

Abstract

Previous work has determined the conditions under which generalized versions of Thurstone's theory of comparative judgment are formally equivalent (i.e., empirically indistinguishable) for choice experiments. This note solves the analogous problem for ranking experiments: It is shown that if two "Generalized Thurstone Models" are equivalent for choice experiments with n alternatives they are also equivalent for ranking experiments with n alternatives, despite the fact that ranking generates many more preference probabilities. This result in turn allows one to determine which Generalized Thurstone Models are "reversible," i.e., satisfy the requirement that regardless of whether the subject ranks from best to worst or from worst to best, rankings that express the same preference order will occur with the same probability.

Some probabilistic models of best, worst, and best–worst choices

“…However, (i) with (38) implies that both the best and the worst choice probabilities satisfy Luce's choice model (Theorem 50, Luce and Suppes, 1965) and then, using (38) again, we obtain that, for each x; y 2 X T; jX jX2; xay, we have B X ðxÞ ¼ W X ðxÞ ¼ 1 jX j (Yellott, 1980). It then follows from the fact that the best, worst and best-worst choice probabilities on T satisfy a concordant best-worst choice model, Definition 14, that BW X ðx; yÞ ¼ 1 jX j Á 1 jX À1j .…”

“…However, and this is important, neither the worst or the best-worst choice probabilities given by that model will then, in general, satisfy the Luce (MNL) choice model when the scale values v z ; z 2 T, are not all equal. Nonetheless, the relationship between the best and the worst choice probabilities in this model is well known, and has lead to much fascinating research (Yellott, 1977(Yellott, , 1980(Yellott, , 1997. Here, we add a result about the representation of best-worst choice probabilities that follows from that earlier work, 5 We include independence as part of the definition as we only consider that case in this paper.…”

Applying sorting algorithms to sensory ranking tests – A proof of concept study

“… Thurstone (1931) describes ranking as “one of the simplest of experimental procedures”, though still noting the high number of paired comparisons needed to contrast every stimulus with all others. There are also theoretical incompatibilities with Thurstonian models and the concept that ranking direction is reversible ( Yellott, 1980 ), i.e. that ranking from best-to-worst will occur with the same probability as when ranking from worst-to-best, which seems theoretically incompatible when ranking is accomplished by carrying out a sequence of totally independent choices.…”

Choosing to Rank

“…To prove this rigorously we will use the following proposition due to Luce, which is simple but widely considered to be surprising (Yellott, 1980;Block and Marschak, 1960;Luce, 1959).…”

“…For example, the MNL model can be derived from Luce's choice axiom (Luce, 1959), commonly referred to as the Independence of Irrelevant Alternatives (IIA). The flexibility of the PCMC and CDM choice models arise largely by eschewing many common choice axioms, but they still exhibit important structure through less rigid choice axioms such as uniform expansion (Yellott, 1980). As part of this work we develop two weaker versions of IIA, local IIA and nested IIA, which we show hold for some PCMC and CDM models.…”