Stopping criteria for active learning of named entity recognition

Laws, Florian; Schütze, Hinrich

doi:10.3115/1599081.1599140

Cited by 56 publications

(57 citation statements)

References 11 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Works exist which attempt to determine operating regions of the active learning cycle to switch between more exploration driven cycles to more exploitation driven cycles [9,31]. A closely related concept is attempting to indicate when active learning has achieved its maximum performance without cross-validation data, referred to as deriving a stopping criteria [14,44,61,71,76,79,80].…”

Section: Related Workmentioning

confidence: 99%

Margin-based active learning for structured predictions

Small

Roth

2010

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

Margin-based active learning remains the most widely used active learning paradigm due to its simplicity and empirical successes. However, most works are limited to binary or multiclass prediction problems, thus restricting the applicability of these approaches to many complex prediction problems where active learning would be most useful. For example, machine learning techniques for natural language processing applications often require combining multiple interdependent prediction problemsgenerally referred to as learning in structured output spaces. In many such application domains, complexity is further managed by decomposing a complex prediction into a sequence of predictions where earlier predictions are used as input to later predictions-commonly referred to as a pipeline model. This work describes methods for extending existing margin-based active learning techniques to these two settings, thus increasing the scope of problems for which active learning can be applied. We empirically validate these proposed active learning techniques by reducing the annotated data requirements on multiple instances of synthetic data, a semantic role labeling task, and a named entity and relation extraction system.

show abstract

Section: Related Workmentioning

confidence: 99%

Margin-based active learning for structured predictions

Small

Roth

2010

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

show abstract

“…To address this problem, some active-learning models are proposed [14,15]. These models showed that manual labeling cost can be reduced without or with only a little degrading of the performances.…”

Section: Previous Workmentioning

confidence: 99%

Low-Cost Implementation of a Named Entity Recognition System for Voice-Activated Human-Appliance Interfaces in a Smart Home

Park

Kim

2018

Sustainability

View full text Add to dashboard Cite

When we develop voice-activated human-appliance interface systems in smart homes, named entity recognition (NER) is an essential tool for extracting execution targets from natural language commands. Previous studies on NER systems generally include supervised machine-learning methods that require a substantial amount of human-annotated training corpus. In the smart home environment, categories of named entities should be defined according to voice-activated devices (e.g., food names for refrigerators and song titles for music players). The previous machine-learning methods make it difficult to change categories of named entities because a large amount of the training corpus should be newly constructed by hand. To address this problem, we present a semi-supervised NER system to minimize the time-consuming and labor-intensive task of constructing the training corpus. Our system uses distant supervision methods with two kinds of auto-labeling processes: auto-labeling based on heuristic rules for single-class named entity corpus generation and auto-labeling based on a pre-trained single-class NER model for multi-class named entity corpus generation. Then, our system improves NER accuracy by using a bagging-based active learning method. In our experiments that included a generic domain that featured 11 named entity classes and a context-specific domain about baseball that featured 21 named entity classes, our system demonstrated good performances in both domains, with F1-measures of 0.777 and 0.958, respectively. Since our system was built from a relatively small human-annotated training corpus, we believe it is a viable alternative to current NER systems in smart home environments.

show abstract

“…Similarly, Laws and Schütze [20] proposed to stop the active learning process when the performance (i.e., confidence), of the learner converges and the gradient of the performance curve approaches 0. Differing from the strategy proposed by Vlachos [35], the gradient based stopping criterion does not require the performance curve to have peaked before stopping.…”

Section: Chapter 2 Related Workmentioning

confidence: 99%

Identifying PCB Contaminated Transformers Through Active Learning

Yeh

Lau

et al. 2013

IEEE Trans. Power Syst.

View full text Add to dashboard Cite

Exposure to polychlorinated biphenyals (PCBs) is hazardous to human health. The United Nations Environment Programme has decreed that nations, including Canada and the US, must eliminate PCB contaminated utility equipment such as transformers by 2025. Sampling, which imposes a non-trivial expenditure, is required to confirm the PCB content of a transformer. For the first time, we apply an iterative machine learning technique known as active learning to construct a PCB transformer identification model that aims to minimize the number of transformers sampled and thus reduce the total cost. In this thesis, we propose a dynamic sampling size algorithm to address two key issues in active learning: the sampling size per iteration and the stopping criterion. The proposed algorithm is evaluated using the real world datasets from BC Hydro in Canada.iii

show abstract

Stopping criteria for active learning of named entity recognition

Cited by 56 publications

References 11 publications

Margin-based active learning for structured predictions

Margin-based active learning for structured predictions

Low-Cost Implementation of a Named Entity Recognition System for Voice-Activated Human-Appliance Interfaces in a Smart Home

Identifying PCB Contaminated Transformers Through Active Learning

Contact Info

Product

Resources

About