Active learning for statistical natural language parsing

Abstract

It is necessary to have a (large) annotated corpus to build a statistical parser. Acquisition of such a corpus is costly and time-consuming. This paper presents a method to reduce this demand using active learning, which selects what samples to annotate, instead of annotating blindly the whole training corpus.Sample selection for annotation is based upon "representativeness" and "usefulness".A model-based distance is proposed to measure the difference of two sentences and their most likely parse trees. Based on this distance, the active learning process analyzes the sample distribution by clustering and calculates the density of each sample to quantify its representativeness. Further more, a sentence is deemed as useful if the existing model is highly uncertain about its parses, where uncertainty is measured by various entropy-based scores.

Active learning for logistic regression: an evaluation

“…Such has happened in the case of part of speech tagging, where the query by committee methods are generalized to apply to hidden Markov models (Dagan and Engelson 1995). In parsing, uncertainty sampling (Hwa 2004) and other heuristic approaches have been applied (Tang et al 2002). A recent trend in the pool-based active learning literature has been to take various approaches, usually uncertainty sampling or query by committee and try to improve performance through additional heuristics.…”

“…A trend of the last ten years (Abe and Mamitsuka 1998;Banko and Brill 2001;Chen et al 2006;Dagan and Engelson 1995;Hwa 2004;Lewis and Gale 1994;McCallum and Nigam 1998;Melville and Mooney 2004;Roy and McCallum 2001;Tang et al 2002) has been to employ heuristic methods of active learning with no explicitly defined objective function. Uncertainty sampling (Lewis and Gale 1994), query by committee (Seung et al 1992), 1 and variants have proven particularly attractive because of their portability across a wide spectrum of machine learning algorithms.…”

“…Uncertainty sampling (Lewis and Gale 1994), query by committee (Seung et al 1992), 1 and variants have proven particularly attractive because of their portability across a wide spectrum of machine learning algorithms. A subtrend in the field has sought to improve performance of heuristics by combining them with secondary heuristics such as: similarity weighting (McCallum and Nigam 1998), interleaving active learning with EM (McCallum and Nigam 1998), interleaving active learning with co-training (Steedman et al 2003), and sampling from clusters (Tang et al 2002), among others.…”

Bootstrapping parsers via syntactic projection across parallel texts

“…One actively researched approach to this problem is to develop weakly supervised algorithms that require less training data, such as active learning (Hermjakob and Mooney 1997;Tang et al 2002;Baldridge and Osborne 2003;Hwa 2004) and co-training (Sarkar 2001;Steedman et al 2003). In this article, we explore an alternative: using parallel text as a means for transferring syntactic knowledge from a resource-rich language to a language with fewer resources.…”

Active Learning for Post-Editing Based Incrementally Retrained MT

“…AL has been successfully applied to many tasks in natural language processing, including parsing (Tang et al, 2002), named entity recognition (Miller et al, 2004), to mention just a few. See (Olsson, 2009) for a comprehensie overview of the application of AL to natural language processing.…”