Relevant knowledge helps in choosing right teacher

Cai, Peng; Gao, Wei; Zhou, Aoying; Wong, Kam-Fai

doi:10.1145/2009916.2009935

Cited by 15 publications

(11 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cai et al. (), for instance, propose a method that integrates domain adaptation and active learning as a way to reduce labeling costs. They first use a QBC scheme built on a mixture of source domain and target domain data to select the most “informative” queries from the target domain.…”

Section: Related Workmentioning

confidence: 99%

“…In an L2R scenario, this measure of disagreement could be, for instance, the Kendall's τ rank correlation coefficient or, as proposed by Cai et al. (), a vote entropy (Dagan & Engelson, ) variation adapted for pairwise ranking. These metrics would allow the measurement of the disagreement of the learners in ranking each query .…”

Section: Two‐stage Active Learning For L2rmentioning

confidence: 99%

“…If we were performing query‐level selection, as in Cai et al. (), we would need to measure the disagreement of each ranking on the query level. Instead, our method works at the document level; that is, we want to measure the disagreement among the learners about the ranking of each document.…”

Section: Two‐stage Active Learning For L2rmentioning

confidence: 99%

“…Active learning techniques have been proposed to help deal with the labeling effort problem in L2R (Cai, Gao, Zhou, & Wong, ; Donmez & Carbonell, , ; Long et al., ; Radlinski & Joachims, ; Silva, Gonçalves, & Veloso, ; Yu, ). The motivation behind active learning is that it may be possible to achieve highly effective learned functions by carefully selecting and labeling instances that are “informative” to the learning algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…In some methods, the most ambiguous, or those instances for which the learner is most uncertain , are selected (Lewis & Gale, ; Tian & Lease, ). In a related strategy, a query‐by‐committee (QBC) strategy is used where competing learners vote on the label of the candidate samples and the one selected for labeling is the one about which the members of the committee most disagree in classifying (Cai et al., ; Seung, Opper, & Sompolinsky, ). Some algorithms select samples that would cause the greatest change to the current learned function (Donmez & Carbonell, ; Settles, Craven, & Ray, ).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Two‐stage active learning method for learning to rank

Silva

Gonçalves

Veloso

2013

Asso for Info Science & Tech

View full text Add to dashboard Cite

Learning to rank (L2R) algorithms use a labeled training set to generate a ranking model that can later be used to rank new query results. These training sets are costly and laborious to produce, requiring human annotators to assess the relevance or order of the documents in relation to a query. Active learning algorithms are able to reduce the labeling effort by selectively sampling an unlabeled set and choosing data instances that maximize a learning function's effectiveness. In this article, we propose a novel two-stage active learning method for L2R that combines and exploits interesting properties of its constituent parts, thus being effective and practical. In the first stage, an association rule active sampling algorithm is used to select a very small but effective initial training set. In the second stage, a query-bycommittee strategy trained with the first-stage set is used to iteratively select more examples until a preset labeling budget is met or a target effectiveness is achieved. We test our method with various LETOR benchmarking data sets and compare it with several baselines to show that it achieves good results using only a small portion of the original training sets.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Two‐stage Active Learning For L2rmentioning

confidence: 99%

Section: Two‐stage Active Learning For L2rmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Two‐stage active learning method for learning to rank

Silva

Gonçalves

Veloso

2013

Asso for Info Science & Tech

View full text Add to dashboard Cite

show abstract

Active learning for ranking with sample density

Cai

Zhang

2015

Inf Retrieval J

View full text Add to dashboard Cite

While ranking is widely used in many online domains such as search engines and recommendation systems, it is non-trivial to label enough data examples to build a high performance machine-learned ranking model. To relieve this problem, active learning has been proposed, which selectively labels the most informative examples. However, data density, which has been proven helpful for data sampling in general, is ignored by most of the existing active learning for ranking studies. In this paper, we propose a novel active learning for ranking framework, generalization error minimization (GEM), which incorporates data density in minimizing generalization error. Concentrating on active learning for search ranking, we employ classical kernel density estimation to infer data density. Considering the unique query-document structure in ranking data, we estimate sample density at both query level and document level. Under the GEM framework, we propose new active learning algorithms at both query level and document level. Experimental results on the LETOR 4.0 data set and a real-world Web search ranking data set from a commercial search engine have demonstrated the effectiveness of the proposed active learning algorithms.

show abstract

Supervised topic models with word order structure for document classification and retrieval learning

Jameel

Lam

2015

Inf Retrieval J

View full text Add to dashboard Cite

One limitation of most existing probabilistic latent topic models for document classification is that the topic model itself does not consider useful side-information, namely, class labels of documents. Topic models, which in turn consider the side-information, popularly known as supervised topic models, do not consider the word order structure in documents. One of the motivations behind considering the word order structure is to capture the semantic fabric of the document. We investigate a low-dimensional latent topic model for document classification. Class label information and word order structure are integrated into a supervised topic model enabling a more effective interaction among such information for solving document classification. We derive a collapsed Gibbs sampler for our model. Likewise, supervised topic models with word order structure have not been explored in document retrieval learning. We propose a novel supervised topic model for document retrieval learning which can be regarded as a pointwise model for tackling the learning-to-rank task. Available relevance assessments and word order structure are integrated into the topic model itself. We conduct extensive experiments on several publicly available benchmark datasets, and show that our model improves upon the state-of-the-art models.

show abstract

Relevant knowledge helps in choosing right teacher

Cited by 15 publications

References 41 publications

A Two‐stage active learning method for learning to rank

A Two‐stage active learning method for learning to rank

Active learning for ranking with sample density

Supervised topic models with word order structure for document classification and retrieval learning

Contact Info

Product

Resources

About