High accuracy retrieval with multiple nested ranker

Matveeva, Irina; Burges, Chris; Burkard, Timo; Laucius, Andy; Wong, Leon

doi:10.1145/1148170.1148246

Cited by 101 publications

(61 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, it is not clear how to train the model to maximize different metrics. Recent work has looked at addressing some of the issues involved in training a RankNet [17].…”

Section: Other Methodsmentioning

confidence: 99%

Linear feature-based models for information retrieval

2007

View full text Add to dashboard Cite

Abstract. There have been a number of linear, feature-based models proposed by the information retrieval community recently. Although each model is presented differently, they all share a common underlying framework. In this paper we explore and discuss the theoretical issues of this framework, including a novel look at the parameter space. We then detail supervised training algorithms that directly maximize the evaluation metric under consideration, such as mean average precision. We present results that show training models in this way can lead to significantly better test set performance compared to other training methods that do not directly maximize the metric. Finally, we show that linear feature-based models can consistently and significantly outperform current state of the art retrieval models with the correct choice of features.

show abstract

“…Therefore, it is not clear how to train the model to maximize different metrics. Recent work has looked at addressing some of the issues involved in training a RankNet [17].…”

Section: Other Methodsmentioning

confidence: 99%

Linear feature-based models for information retrieval

2007

View full text Add to dashboard Cite

show abstract

“…[59] extends the well-studied SVM selecting sampling technique in classification for LTR. [31] proposes a multiple nested ranker approach to re-rank the top scoring documents of the result list, in which RankNet is applied to learn a new ranking at each iteration.…”

Section: Related Workmentioning

confidence: 99%

Learning Ranking Functions For Information Retrieval Using Layered Multi-Population Genetic Programming

Yeh

Lin²

2017

MJCS

View full text Add to dashboard Cite

Ranking plays a key role in many applications, such as document retrieval, recommendation, question answering, and machine translation. In practice, a ranking function (or model) is exploited to determine the rank-order relations between objects, with respect to a particular criterion. In this paper, a layered multipopulation genetic programming based method, known as RankMGP, is proposed to learn ranking functions for document retrieval by incorporating various types of retrieval models into a singular one with high effectiveness. RankMGP represents a potential solution (i.e., a ranking function) as an individual in a population of genetic programming and aims to directly optimize information retrieval evaluation measures in the evolution process. Overall, RankMGP consists of a set of layers and a sequential workflow running through the layers. In one layer, multiple populations evolve independently to generate a set of the best individuals. When the evolution process is completed, a new training dataset is created using the best individuals and the input training set of the layer. Then, the populations in the next layer evolve with the new training dataset. In the final layer, the best individual is obtained as the output ranking function. The proposed method is evaluated using the LETOR datasets and is found to be superior to classical information retrieval models, such as Okapi BM25. It is also statistically competitive with the state-of-the-art methods, including Ranking SVM, ListNet, AdaRank and RankBoost.Keywords: learning to rank for information retrieval, ranking function, supervised learning, layered multipopulation genetic programming, LAGEP, LETOR INTRODUCTIONOne central problem of information retrieval (IR) is the issue of determining which documents are relevant to the user's information needs, and which are not (i.e., finding the potentially relevant documents) [3]. In practice, it is usually addressed as a ranking problem, whose goal is, according to the degree of relevance (or similarity) between each document and the user's query, to define a total order of documents that ranks relevant documents in higher positions on the retrieved list than irrelevant ones.Traditional IR models, including the Boolean model, the vector space model, and the probabilistic model, are developed based on the bag-of-words model. In short, a document is decomposed into keywords (i.e., index terms) and a ranking function (or retrieval function) is defined to associate a relevance degree with the document and query [3]. The aforementioned models are typically realized in an unsupervised manner and thus, the parameters of underlying ranking functions are usually tuned empirically. However, manual tuning suffers high costs and sometimes leads to over-fitting, especially when the functions are carefully tuned to fit particular needs [30].Nowadays, as increasingly many IR results are accompanied by relevance judgments (e.g., query and clickthrough logs), supervised learning-based methods, referred to as "learning t...

show abstract

“…RankNet [11] employs a neural network to learn ranking functions by minimizing the cross entropy that measures the difference between the modeled probabilities of pair preferences and the ground truth ones. Besides, LambdaRank [16], FRank [17] and Multiple-nestedranker [18] are all pairwise approaches.…”

Section: Pairwise Approachesmentioning

confidence: 99%

BLM-Rank: A Bayesian Linear Method for Learning to Rank and Its GPU Implementation

Guo

Chu

et al. 2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYRanking as an important task in information systems has many applications, such as document/webpage retrieval, collaborative filtering and advertising. The last decade has witnessed a growing interest in the study of learning to rank as a means to leverage training information in a system. In this paper, we propose a new learning to rank method, i.e. BLM-Rank, which uses a linear function to score samples and models the pairwise preference of samples relying on their scores under a Bayesian framework. A stochastic gradient approach is adopted to maximize the posterior probability in BLM-Rank. For industrial practice, we have also implemented the proposed algorithm on Graphic Processing Unit (GPU). Experimental results on LETOR have demonstrated that the proposed BLM-Rank method outperforms the state-of-the-art methods, including RankSVM-Struct, RankBoost, AdaRank-NDCG, AdaRank-MAP and ListNet. Moreover, the results have shown that the GPU implementation of the BLM-Rank method is ten-to-eleven times faster than its CPU counterpart in the training phase, and one-to-four times faster in the testing phase.

show abstract

High accuracy retrieval with multiple nested ranker

Cited by 101 publications

References 11 publications

Linear feature-based models for information retrieval

Linear feature-based models for information retrieval

Learning Ranking Functions For Information Retrieval Using Layered Multi-Population Genetic Programming

BLM-Rank: A Bayesian Linear Method for Learning to Rank and Its GPU Implementation

Contact Info

Product

Resources

About