Sample Selection for Statistical Parsing

Hwa, Rebecca

doi:10.1162/0891201041850894

Cited by 96 publications

(86 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use this algorithm to compute the true sequence entropy (3) for active learning in a constant-time factor of Viterbi's complexity. Hwa (2004) employed a similar approach for active learning with probabilistic context-free grammars.…”

Section: Uncertainty Samplingmentioning

confidence: 99%

An analysis of active learning strategies for sequence labeling tasks

Settles

Craven

2008

Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08

791

634

View full text Add to dashboard Cite

Active learning is well-suited to many problems in natural language processing, where unlabeled data may be abundant but annotation is slow and expensive. This paper aims to shed light on the best active learning approaches for sequence labeling tasks such as information extraction and document segmentation. We survey previously used query selection strategies for sequence models, and propose several novel algorithms to address their shortcomings. We also conduct a large-scale empirical comparison using multiple corpora, which demonstrates that our proposed methods advance the state of the art.

show abstract

Section: Uncertainty Samplingmentioning

confidence: 99%

An analysis of active learning strategies for sequence labeling tasks

Settles

Craven

2008

Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08

791

634

View full text Add to dashboard Cite

show abstract

“…A trend of the last ten years (Abe and Mamitsuka 1998;Banko and Brill 2001;Chen et al 2006;Dagan and Engelson 1995;Hwa 2004;Lewis and Gale 1994;McCallum and Nigam 1998;Melville and Mooney 2004;Roy and McCallum 2001;Tang et al 2002) has been to employ heuristic methods of active learning with no explicitly defined objective function. Uncertainty sampling (Lewis and Gale 1994), query by committee (Seung et al 1992), 1 and variants have proven particularly attractive because of their portability across a wide spectrum of machine learning algorithms.…”

Section: Background and Related Workmentioning

confidence: 99%

Active learning for logistic regression: an evaluation

2007

View full text Add to dashboard Cite

Which active learning methods can we expect to yield good performance in learning binary and multi-category logistic regression classifiers? Addressing this question is a natural first step in providing robust solutions for active learning across a wide variety of exponential models including maximum entropy, generalized linear, log-linear, and conditional random field models. For the logistic regression model we re-derive the variance reduction method known in experimental design circles as 'A-optimality.' We then run comparisons against different variations of the most widely used heuristic schemes: query by committee and uncertainty sampling, to discover which methods work best for different classes of problems and why. We find that among the strategies tested, the experimental design methods are most likely to match or beat a random sample baseline. The heuristic alternatives produced mixed results, with an uncertainty sampling variant called margin sampling and a derivative method called QBB-MM providing the most promising performance at very low computational cost. Computational running times of the experimental design methods were a bottleneck to the evaluations. Meanwhile, evaluation of the heuristic methods lead to an accumulation of negative results. We explore alternative evaluation design parameters to test whether these negative results are merely an artifact of settings where experimental design methods can be applied. The results demonstrate a need for improved active learning methods that will provide reliable performance at a reasonable computational cost.

show abstract

“…Active Learning (AL) (Settles, 2009;Settles and Craven, 2008;Hwa, 2004;Tong and Koller, 2001) builds another important set of related work. Our method is inspired by uncertainty sampling.…”

Section: Related Workmentioning

confidence: 99%

Iterative Constrained Clustering for Subjectivity Word Sense Disambiguation

Akkaya

Wiebe

Mihalcea

2014

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Subjectivity word sense disambiguation (SWSD) is a supervised and applicationspecific word sense disambiguation task disambiguating between subjective and objective senses of a word. Not surprisingly, SWSD suffers from the knowledge acquisition bottleneck. In this work, we use a "cluster and label" strategy to generate labeled data for SWSD semiautomatically. We define a new algorithm called Iterative Constrained Clustering (ICC) to improve the clustering purity and, as a result, the quality of the generated data. Our experiments show that the SWSD classifiers trained on the ICC generated data by requiring only 59% of the labels can achieve the same performance as the classifiers trained on the full dataset.

show abstract

Sample Selection for Statistical Parsing

Cited by 96 publications

References 24 publications

An analysis of active learning strategies for sequence labeling tasks

An analysis of active learning strategies for sequence labeling tasks

Active learning for logistic regression: an evaluation

Iterative Constrained Clustering for Subjectivity Word Sense Disambiguation

Contact Info

Product

Resources

About