Choosing to Rank

Abstract

Ranking data arises in a wide variety of application areas but remains difficult to model, learn from, and predict. Datasets often exhibit multimodality, intransitivity, or incomplete rankingsparticularly when generated by humans-yet popular probabilistic models are often too rigid to capture such complexities. In this work we leverage recent progress on similar challenges in discrete choice modeling to form flexible and tractable choice-based models for ranking data. We study choice representations, maps from rankings (complete or top-k) to collections of choices, as a way of forming ranking models from choice models. We focus on the repeated selection (RS) choice representation, first used to form the Plackett-Luce ranking model from the conditional multinomial logit choice model. We fully characterize, for a prime number of alternatives, the choice representations that admit ranking distributions with unit normalization, a desirably property that greatly simplifies maximum likelihood estimation. We further show that only specific minor variations on repeated selection exhibit this property. Our choice-based ranking models provide higher out-of-sample likelihood when compared to Plackett-Luce and Mallows models on a broad collection of ranking tasks including food preferences, ranked-choice elections, car racing, and search engine relevance tasks.

Predicting Choice with Set-Dependent Aggregation

“…Although recent progress has alleviated some difficulties [25,3,6], applying these models in practice remains difficult. Recent works have proposed other probabilistic models that deviate from IIA, by modeling pairwise utilities [32], k th -order interactions [31,34], and general subset relations [4]. These, however, do not optimize for predictive accuracy, and rarely apply to complex choice settings with many items and sparse choice sets.…”

“…From a practical point of view, this is discouraging, as there is ample empirical evidence of regular and consistent violations of IIA in real choice data (e.g., [33]). This has led to a surge of interest in machine learning models that go beyond IIA [25,27,3,31,28,32,6,34,30].…”