Ranking Model Averaging: Ranking Based on Model Averaging

Abstract

Ranking problems are commonly encountered in practical applications, including order priority ranking, wine quality ranking, and piston slap noise performance ranking. The responses of these ranking applications are often considered as continuous responses, and there is uncertainty on which scoring function is used to model the responses. In this paper, we address the scoring function uncertainty of continuous response ranking problems by proposing a ranking model averaging (RMA) method. With a set of candidate models varied by scoring functions, RMA assigns weights for each model determined by a K-fold crossvalidation criterion based on pairwise loss. We provide two main theoretical properties for RMA. First, we prove that the averaging ranking predictions of RMA are asymptotically optimal in achieving the lowest possible ranking risk. Second, we provide a bound on the difference between the empirical RMA weights and theoretical optimal ones, and we show that RMA weights are consistent. Simulation results validate RMA superiority over competing methods in reducing ranking risk. Moreover, when applied to empirical examples—order priority, wine quality, and piston slap noise—RMA shows its effectiveness in building accurate ranking systems. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: This research was supported by the Beijing Municipal Natural Science Foundation [Grant 1222002], the National Natural Science Foundation of China [Grants 12071457, 12201018, 12301364, 71925007, 72091212, and 72273120], National Quality Infrastructure [Grant 2022YFF0609903], the Chinese Academy of Sciences Project for Young Scientists in Basic Research [Grant YSBR-008], and the Natural Science Foundation of Anhui Province [Grant 2308085QA09]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://doi.org/10.1287/ijoc.2023.0257 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0257 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .