Reliability-aware Dynamic Feature Composition for Name Tagging

Lin, Ying; Liu, Liyuan; Ji, Heng; Yu, Dong

doi:10.18653/v1/p19-1016

Cited by 22 publications

(9 citation statements)

References 27 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Empirical results demonstrate the effectiveness of our proposed method. In future work, we plan to apply the proposed method to other applications such as Named Entity Recognition (Reimers & Gurevych, 2017;Lin et al, 2019). Another interesting direction to pursue is to adapt the choice of β based on the variance estimation of different parameters, i.e., use a larger β for parameters with a larger variance.…”

Section: Discussionmentioning

confidence: 99%

On the Variance of the Adaptive Learning Rate and Beyond

Liu¹,

Jiang²,

He³

et al. 2019

Preprint

Self Cite

309

296

View full text Add to dashboard Cite

The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam. Here, we study its mechanism in details. Pursuing the theory behind warmup, we identify a problem of the adaptive learning rate (i.e., it has problematically large variance in the early stage), suggest warmup works as a variance reduction technique, and provide both empirical and theoretical evidence to verify our hypothesis. We further propose RAdam, a new variant of Adam, by introducing a term to rectify the variance of the adaptive learning rate. Extensive experimental results on image classification, language modeling, and neural machine translation verify our intuition and demonstrate the effectiveness and robustness of our proposed method. 1 * Work was done during an internship at Microsoft. † Work was done during an internship at Microsoft.

show abstract

Section: Discussionmentioning

confidence: 99%

On the Variance of the Adaptive Learning Rate and Beyond

Liu¹,

Jiang²,

He³

et al. 2019

Preprint

Self Cite

309

296

View full text Add to dashboard Cite

show abstract

“…In the NLP domain, NER is usually considered as a sequence labeling problem Lin et al, 2019b;Cao et al, 2019). With well-designed features, CRF-based models have achieved the leading performance (Lafferty et al, 2001;Finkel et al, 2005;Liu et al, 2011).…”

Section: Related Workmentioning

confidence: 99%

“…Named entity recognition (NER) (Sang and De Meulder, 2003) is one fundamental task for natural language processing (NLP), due to its wide application in information extraction and data mining (Lin et al, 2019b;Cao et al, 2019). Traditionally, NER is presented as a sequence labeling problem and widely solved by conditional random field (CRF) based models (Lafferty et al, 2001).…”

Section: Introductionmentioning

confidence: 99%

A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition

Lin²,

Zhang

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Research on overlapped and discontinuous named entity recognition (NER) has received increasing attention. The majority of previous work focuses on either overlapped or discontinuous entities. In this paper, we propose a novel span-based model that can recognize both overlapped and discontinuous entities jointly. The model includes two major steps. First, entity fragments are recognized by traversing over all possible text spans, thus, overlapped entities can be recognized. Second, we perform relation classification to judge whether a given pair of entity fragments to be overlapping or succession. In this way, we can recognize not only discontinuous entities, and meanwhile doubly check the overlapped entities. As a whole, our model can be regarded as a relation extraction paradigm essentially. Experimental results on multiple benchmark datasets (i.e., CLEF, GENIA and ACE05) show that our model is highly competitive for overlapped and discontinuous NER.

show abstract

“…We take ground truth text entity mentions as input following (Ji and Grishman, 2008) during training, and obtain testing entity mentions using a named entity extractor (Lin et al, 2019).…”

Section: Text Event Extractionmentioning

confidence: 99%

Cross-media Structured Common Space for Multimedia Event Extraction

Li¹,

Zareian²,

Zeng³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

We introduce a new task, MultiMedia Event Extraction (M 2 E 2 ), which aims to extract events and their arguments from multimedia documents. We develop the first benchmark and collect a dataset of 245 multimedia news articles with extensively annotated events and arguments. 1 We propose a novel method, Weakly Aligned Structured Embedding (WASE), that encodes structured representations of semantic information from textual and visual data into a common embedding space. The structures are aligned across modalities by employing a weakly supervised training strategy, which enables exploiting available resources without explicit cross-media annotation. Compared to unimodal state-of-the-art methods, our approach achieves 4.0% and 9.8% absolute F-score gains on text event argument role labeling and visual event extraction. Compared to stateof-the-art multimedia unstructured representations, we achieve 8.3% and 5.0% absolute Fscore gains on multimedia event extraction and argument role labeling, respectively. By utilizing images, we extract 21.4% more event mentions than traditional text-only methods.

show abstract

Reliability-aware Dynamic Feature Composition for Name Tagging

Cited by 22 publications

References 27 publications

On the Variance of the Adaptive Learning Rate and Beyond

On the Variance of the Adaptive Learning Rate and Beyond

A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition

Cross-media Structured Common Space for Multimedia Event Extraction

Contact Info

Product

Resources

About