Weakly Supervised Sequence Tagging from Noisy Rules

Safranchik, Esteban; Luo, Sheng; Bach, Stephen H.

doi:10.1609/aaai.v34i04.6009

Cited by 83 publications

(112 citation statements)

References 32 publications

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…There are also some works that focus on redefining NER as a different problem for reducing the need of hand-labeled training data. For example, Linking Rules (Safranchik et al, 2020) based on votes recognize entities through whether adjacent elements in a sequence belong to the same class; Lin et al propose a new effective proxy of human explanation, "entity triggers", for encouraging label-efficient learning of NER models.…”

Section: Related Workmentioning

confidence: 99%

Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition

Zeng¹,

Li²,

Zhai³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Past progress on neural models has proven that named entity recognition is no longer a problem if we have enough labeled data. However, collecting enough data and annotating them are labor-intensive, time-consuming, and expensive. In this paper, we decompose the sentence into two parts: entity and context, and rethink the relationship between them and model performance from a causal perspective. Based on this, we propose the Counterfactual Generator, which generates counterfactual examples by the interventions on the existing observational examples to enhance the original dataset. Experiments across three datasets show that our method improves the generalization ability of models under limited observational examples. Besides, we provide a theoretical foundation by using a structural causal model to explore the spurious correlations between input features and output labels. We investigate the causal effects of entity or context on model performance under both conditions: the non-augmented and the augmented. Interestingly, we find that the non-spurious correlations are more located in entity representation rather than context representation. As a result, our method eliminates part of the spurious correlations between context representation and output labels. The code is available at https://github.com/xijiz/cfgen.

show abstract

Section: Related Workmentioning

confidence: 99%

Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition

Zeng¹,

Li²,

Zhai³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…The approach most closely related to this paper is Safranchik et al (2020), which describe a similar weak supervision framework for sequence labelling based on an extension of HMMs called linked hidden Markov models. The authors introduce a new type of noisy rules, called linking rules, to determine how sequence elements should be grouped into spans of same tag.…”

Section: Weak Supervisionmentioning

confidence: 99%

Named Entity Recognition without Labelled Data: A Weak Supervision Approach

Lison

Barnes

Hubin

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Named Entity Recognition (NER) performance often degrades rapidly when applied to target domains that differ from the texts observed during training. When in-domain labelled data is available, transfer learning techniques can be used to adapt existing NER models to the target domain. But what should one do when there is no hand-labelled data for the target domain? This paper presents a simple but powerful approach to learn NER models in the absence of labelled data through weak supervision. The approach relies on a broad spectrum of labelling functions to automatically annotate texts from the target domain. These annotations are then merged together using a hidden Markov model which captures the varying accuracies and confusions of the labelling functions. A sequence labelling model can finally be trained on the basis of this unified annotation. We evaluate the approach on two English datasets (CoNLL 2003 and news articles from Reuters and Bloomberg) and demonstrate an improvement of about 7 percentage points in entity-level F 1 scores compared to an out-of-domain neural NER model.

show abstract

“…TagRuler implements atomic rules which are specifically for span annotation tasks. Following previous works on rule-based labeling models [9,19], we focus on capturing semantic, syntactic, and entity-type information of the spans. 2.2.1 Contextual similarity rules with neural embeddings.…”

Section: Synthesizermentioning

confidence: 99%

TagRuler: Interactive Tool for Span-Level Data Programming by Demonstration

Choi

Evensen

Demiralp

et al. 2021

Companion Proceedings of the Web Conference 2021

View full text Add to dashboard Cite

Despite rapid developments in the field of machine learning research, collecting high quality labels for supervised learning remains a bottleneck for many applications. This difficulty is exacerbated by the fact that state-of-the art models for NLP tasks are becoming deeper and more complex, often increasing the amount of training data required even for fine-tuning. Weak supervision methods, including data programming, address this problem and reduce the cost of label collection by using noisy label sources for supervision. However until recently, data programming was only accessible to users who knew how to program. In order to bridge this gap, the Data Programming by Demonstration framework was proposed to facilitate the automatic creation of labeling functions based on a few examples labeled by a domain expert. This framework has proven successful for generating high accuracy labeling models for document classification. In this work, we extend the DPBD framework to span-level annotation tasks, arguably one of the most time consuming NLP labeling tasks. We built a novel tool, TagRuler, that makes it easy for annotators to build span-level labeling functions without programming and encourages them to explore trade-offs between different labeling models and active learning strategies. We empirically demonstrated that an annotator could achieve a higher F1 score using the proposed tool compared to manual labeling for different span-level annotation tasks. CCS CONCEPTS• Information systems → Web applications.

show abstract

Weakly Supervised Sequence Tagging from Noisy Rules

Cited by 83 publications

References 32 publications

Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition

Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition

Named Entity Recognition without Labelled Data: A Weak Supervision Approach

TagRuler: Interactive Tool for Span-Level Data Programming by Demonstration

Contact Info

Product

Resources

About