Active Sampling for Rank Learning via Optimizing the Area under the ROC Curve

Dönmez, Pınar; Carbonell, Jaime G.

doi:10.1007/978-3-642-00958-7_10

Cited by 28 publications

(29 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to the approach developed in this paper, approaches combining active learning [4,23] with exploration strategies from RL could be developed.…”

Section: Resultsmentioning

confidence: 99%

Balancing Exploration and Exploitation in Learning to Rank Online

Hofmann

Whiteson

Rijke

2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. As retrieval systems become more complex, learning to rank approaches are being developed to automatically tune their parameters. Using online learning to rank approaches, retrieval systems can learn directly from implicit feedback, while they are running. In such an online setting, algorithms need to both explore new solutions to obtain feedback for effective learning, and exploit what has already been learned to produce results that are acceptable to users. We formulate this challenge as an exploration-exploitation dilemma and present the first online learning to rank algorithm that works with implicit feedback and balances exploration and exploitation. We leverage existing learning to rank data sets and recently developed click models to evaluate the proposed algorithm. Our results show that finding a balance between exploration and exploitation can substantially improve online retrieval performance, bringing us one step closer to making online learning to rank work in practice.

show abstract

“…In addition to the approach developed in this paper, approaches combining active learning [4,23] with exploration strategies from RL could be developed.…”

Section: Resultsmentioning

confidence: 99%

Balancing Exploration and Exploitation in Learning to Rank Online

Hofmann

Whiteson

Rijke

2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The key of this algorithm is constructing an appropriate image model. Supervised learning re-ranking [14][15][16][17][18] determines the sample training classifier from initial results and then accomplishes re-ranking with the classifier. Supervised learning re-ranking algorithms include the pseudo-relevance feedback (PRF) algorithm used by Yan et al [19] of Carnegie Mellon University, which was also the conceptual test modal algorithm used by Kennedy and Chang et al [20] of Columbia University.…”

Section: State Of the Artmentioning

confidence: 99%

Analysis on Digital Image Re-Ranking Algorithm based on Multi-feature Fusion

Huanping¹,

Liu²,

Huojie³

2017

JESTR

View full text Add to dashboard Cite

Digital image re-ranking algorithms directly retrieve images. Without text messages or images in these algorithms, a semantic gap between the visual feature and high-level semantics of the image exist, which will affect the sequencing performance of retrieval results. In this study, a digital image re-ranking algorithm based on multi-feature fusion was proposed to eliminate the effects of semantic gap on image re-ranking. This algorithm explored the deep relationship between the visual feature and semantic attributes of digital images by combining the background visual feature and semantic attributes. The mapping dictionary between visual features and semantic attributes was also constructed from the perspective of transition probability. The relevant weight components of visual feature in each dimension were calculated based on the query of context between the visual features in the expanded set and the relevance analysis of semantic attributes. Subsequently, the relevant fraction calculation formula was then established from diversity and relevance, and the relevant fraction of each digital image was sequenced. Finally, the proposed algorithm was verified by the comparison analysis using visual rank algorithm and pseudo-relevance feedback (PRF) algorithm. Results demonstrated that the non-interpolating average precision (AP) of the proposed algorithm in public databases Holiday, Oxford 5k, and Paris are 0.8456, 0.5412 and 0.7075, which are 2.4%, 3.29%, and 3.61%, and 0.86%, 4.24% and 6.4% higher than those of visual rank algorithm and PRF algorithm, respectively. The AP of the proposed algorithm increased significantly. Research results showed that the elimination of semantic gap could improve digital image re-ranking to some extent. Related conclusions could provide technical support to network-based image retrieval.

show abstract

“…These methods, however, are intended to be used during training only since some of them do not aim to retrieve more relevant documents but rather those that are "interesting" to the learning process. Donmez and Carbonell [24] propose to further sub-sample the initial sample of documents using active learning in order to focus the learning process on the most informative training instances, which is again on the learning side of the framework. Our Stage A model, on the contrary, aims to provide more relevant documents not only for the Stage B model to learn from (training) but also to re-rank (run-time).…”

Section: Related Workmentioning

confidence: 99%

Two-Stage Learning to Rank for Information Retrieval

Dang

Bendersky

Croft

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Current learning to rank approaches commonly focus on learning the best possible ranking function given a small fixed set of documents. This document set is often retrieved from the collection using a simple unsupervised bag-of-words method, e.g. BM25. This can potentially lead to learning a sub-optimal ranking, since many relevant documents may be excluded from the initially retrieved set. In this paper we propose a novel two-stage learning framework to address this problem. We first learn a ranking function over the entire retrieval collection using a limited set of textual features including weighted phrases, proximities and expansion terms. This function is then used to retrieve the best possible subset of documents over which the final model is trained using a larger set of query-and document-dependent features. Empirical evaluation using two web collections unequivocally demonstrates that our proposed two-stage framework, being able to learn its model from more relevant documents, outperforms current learning to rank approaches.

show abstract

Active Sampling for Rank Learning via Optimizing the Area under the ROC Curve

Cited by 28 publications

References 21 publications

Balancing Exploration and Exploitation in Learning to Rank Online

Balancing Exploration and Exploitation in Learning to Rank Online

Analysis on Digital Image Re-Ranking Algorithm based on Multi-feature Fusion

Two-Stage Learning to Rank for Information Retrieval

Contact Info

Product

Resources

About