A Boosting Approach for Learning to Rank Using SVD with Partially Labeled Data

Lin, Yuan; Lin, Hongfei; Yang, Zhihao; Su, Sui

doi:10.1007/978-3-642-04769-5_29

Cited by 2 publications

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

A cross-benchmark comparison of 87 learning to rank methods

Tax

Bockting²,

Hiemstra

2015

Information Processing & Management

View full text Add to dashboard Cite

Learning to rank is an increasingly important scientific field that comprises the use of machine learning for the ranking task. New learning to rank methods are generally evaluated on benchmark test collections. However, comparison of learning to rank methods based on evaluation results is hindered by nonexistence of a standard set of evaluation benchmark collections. In this paper we propose a way to compare learning to rank methods based on a sparse set of evaluation results on a set of benchmark datasets. Our comparison methodology consists of two components: 1) Normalized Winning Number, which gives insight in the ranking accuracy of the learning to rank method, and 2) Ideal Winning Number, which gives insight in the degree of certainty concerning its ranking accuracy. Evaluation results of 87 learning to rank methods on 20 wellknown benchmark datasets are collected through a structured literature search. ListNet, SmoothRank, FenchelRank, FSMRank, LRUF and LARF are Pareto optimal learning to rank methods in the Normalized Winning Number and Ideal Winning Number dimensions, listed in increasing order of Normalized Winning Number and decreasing order of Ideal Winning Number.

show abstract

A cross-benchmark comparison of 87 learning to rank methods

Tax

Bockting²,

Hiemstra

2015

Information Processing & Management

View full text Add to dashboard Cite

show abstract

Pseudo test collections for learning web search ranking functions

Asadi

Metzler

Elsayed

et al. 2011

Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Test collections are the primary drivers of progress in information retrieval. They provide yardsticks for assessing the effectiveness of ranking functions in an automatic, rapid, and repeatable fashion and serve as training data for learning to rank models. However, manual construction of test collections tends to be slow, labor-intensive, and expensive. This paper examines the feasibility of constructing web search test collections in a completely unsupervised manner given only a large web corpus as input. Within our proposed framework, anchor text extracted from the web graph is treated as a pseudo query log from which pseudo queries are sampled. For each pseudo query, a set of relevant and non-relevant documents are selected using a variety of webspecific features, including spam and aggregated anchor text weights. The automatically mined queries and judgments form a pseudo test collection that can be used for training ranking functions. Experiments carried out on TREC web track data show that learning to rank models trained using pseudo test collections outperform an unsupervised ranking function and are statistically indistinguishable from a model trained using manual judgments, demonstrating the usefulness of our approach in extracting reasonable quality training data "for free".

show abstract

A Boosting Approach for Learning to Rank Using SVD with Partially Labeled Data

Cited by 2 publications

References 13 publications

A cross-benchmark comparison of 87 learning to rank methods

A cross-benchmark comparison of 87 learning to rank methods

Pseudo test collections for learning web search ranking functions

Contact Info

Product

Resources

About