Efficient rank aggregation using partial data

Abstract

The need to rank items based on user input arises in many practical applications such as elections, group decision making and recommendation systems. The primary challenge in such scenarios is to decide on a global ranking based on partial preferences provided by users. The standard approach to address this challenge is to ask users to provide explicit numerical ratings (cardinal information) of a subset of the items. The main appeal of such an approach is the ease of aggregation. However, the rating scale as well as the individual ratings are often arbitrary and may not be consistent from one user to another. A more natural alternative to numerical ratings requires users to compare pairs of items (ordinal information). On the one hand, such comparisons provide an "absolute" indicator of the user's preference. On the other hand, it is often hard to combine or aggregate these comparisons to obtain a consistent global ranking. In this work, we provide a tractable framework for utilizing comparison data as well as first-order marginal information (see Section 2) for the purpose of ranking. We treat the available information as partial samples from an unknown distribution over permutations. We then reduce ranking problems of interest to performing inference on this distribution. Specifically, we consider the problems of (a) finding an aggregate ranking of n items, (b) learning the mode of the distribution, and (c) identifying the top k items. For many of these problems, we provide efficient algorithms to infer the ranking directly from the data without the need to estimate the underlying distribution. In other cases, we use the Principle of Maximum Entropy to devise a concise parameterization of a distribution consistent with observations using only O(n 2 ) parameters, where n is the number of items in question. We propose a distributed, iterative algorithm for estimating the parameters of the distribution. We establish the correctness of the algorithm and identify its rate of convergence explicitly.

Spectral method and regularized MLE are both optimal for top-$K$ ranking

“…Moreover, it is not obvious whether the spectral method alone or the regularized MLE alone can achieve the minimal sample complexity in the general κ regime. It is possible that one needs to first screen out those items with extremely high or low scores using methods like Borda count (Ammar and Shah, 2012), as advocated by (Negahban et al, 2017a;Chen and Suh, 2015;Jang et al, 2016). All in all, finding tight upper bounds for general κ remains an open question.…”

Weighted aggregation of partial rankings using Ant Colony Optimization

“…Ammar and Shah [22] consider partial data in the form of first-order or comparison marginals. They treat this information as partial samples from an unknown distribution over permutations and provide an efficient algorithm for finding an aggregate complete ranking directly from the data without first learning the underlying distribution; this is an appealing feature for designing large-scale ranking systems such as recommendation systems.…”

Ordinal UNLOC: Target Localization With Noisy and Incomplete Distance Measures

“…The problem of inferring the ranking or scoring of a set of items from their pairwise comparisons is commonly known as "rank aggregation" [39], [40], interpreting each pairwise comparison as assigning a local ranking between two items, with the goal of obtaining an aggregated global ranking that preserves these local rankings as much as possible. Several methods are available for solving a rank aggregation problem, most of which compute a score for each item based on the collection of ordinal data [41].…”