Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation

Yang, Huiyun; Huang, Shujian; Dai, Xinyu; Chen, Jiajun

doi:10.18653/v1/d19-1429

Cited by 11 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al (2022b) explored the semantic transfer of labels simultaneously in the entity span and the type space, thus achieving smaller discrepancies in the cross-domain transfer. The methods of model parameter transfer aim to share model parameters between different domains through knowledge distillation (Yang et al 2019;Nguyen, Gelli, and Poria 2021;Zhang et al 2021), domain prediction tasks (Lin and Lu 2018;Zhou et al 2019;Jia and Zhang 2020; or generative methods (Jia, Liang, and Zhang 2019;Chen et al 2021;). However, these methods neglect to request more source domain data to explicitly and implicitly transfer knowledge to the target domain.…”

Section: Unrecognized Entitymentioning

confidence: 99%

Three Heads Are Better than One: Improving Cross-Domain NER with Progressive Decomposed Network

Hu,

Hong,

Jiang

et al. 2024

AAAI

View full text Add to dashboard Cite

Cross-domain named entity recognition (NER) tasks encourage NER models to transfer knowledge from data-rich source domains to sparsely labeled target domains. Previous works adopt the paradigms of pre-training on the source domain followed by fine-tuning on the target domain. However, these works ignore that general labeled NER source domain data can be easily retrieved in the real world, and soliciting more source domains could bring more benefits. Unfortunately, previous paradigms cannot efficiently transfer knowledge from multiple source domains. In this work, to transfer multiple source domains' knowledge, we decouple the NER task into the pipeline tasks of mention detection and entity typing, where the mention detection unifies the training object across domains, thus providing the entity typing with higher-quality entity mentions. Additionally, we request multiple general source domain models to suggest the potential named entities for sentences in the target domain explicitly, and transfer their knowledge to the target domain models through the knowledge progressive networks implicitly. Furthermore, we propose two methods to analyze in which source domain knowledge transfer occurs, thus helping us judge which source domain brings the greatest benefit. In our experiment, we develop a Chinese cross-domain NER dataset. Our model improved the F1 score by an average of 12.50% across 8 Chinese and English datasets compared to models without source domain data.

show abstract

Section: Unrecognized Entitymentioning

confidence: 99%

Three Heads Are Better than One: Improving Cross-Domain NER with Progressive Decomposed Network

Hu,

Hong,

Jiang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Prior work on domain adaptation for coref has focused on a single dataset and often with non-neural models. Yang et al (2012) use an adaptive ensemble which adjusts members per document. Meanwhile, Zhao and Ng (2014) use an active learning approach to adapt a feature-based coref model to be on par with one trained from scratch while using far less data.…”

Section: Arrau (News)mentioning

confidence: 99%

Moving on from OntoNotes: Coreference Resolution Model Transfer

Xia

Durme

2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Academic neural models for coreference resolution (coref) are typically trained on a single dataset, OntoNotes, and model improvements are benchmarked on that same dataset. However, real-world applications of coref depend on the annotation guidelines and the domain of the target dataset, which often differ from those of OntoNotes. We aim to quantify transferability of coref models based on the number of annotated documents available in the target dataset. We examine eleven target datasets and find that continued training is consistently effective and especially beneficial when there are few target documents. We establish new benchmarks across several datasets, including state-of-the-art results on PreCo.

show abstract

“…These two-stage methods allow using large-scale unlabeled data in pre-training and small labeled data in fine-tuning. In order to adapt to specific tasks or domain, variants of BERT are proposed including small and practical BERT (Tsai et al, 2019;Lan et al, 2020b;Jiao et al, 2020), domain adaptive BERT (Yang et al, 2019a;Gururangan et al, 2020), and task adaptive BERT Xue et al, 2020;Jia et al, 2020). Our work performs further pre-training on BERT and proposes task-aware training objectives to improve NER.…”

Section: Two-stage Training Paradigm For Nlpmentioning

confidence: 99%

Toward Fully Exploiting Heterogeneous Corpus:A Decoupled Named Entity Recognition Model with Two-stage Training

Hu¹,

Zhu²,

Zhang³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Named Entity Recognition (NER) is a fundamental and widely used task in natural language processing (NLP), which is generally trained on the human-annotated corpus. However, data annotation is costly and timeconsuming, which restricts its scale and further leads to the performance bottleneck of NER models. In reality, we can conveniently collect large-scale entity dictionaries and distantly supervised data. However, the collected dictionaries are lack of semantic context and the distantly supervised training instances contain large noise, which will bring uncertain effects to NER models when directly incorporated into the high-quality training set. To address the above issue, we propose a BERT-based decoupled NER model with two-stage training to appropriately take advantage of the heterogeneous corpus, including dictionaries, distantly supervised instances, and human-annotated instances. Our decoupled model consists of a Mention-BERT and a Context-BERT to respectively learn from the context-deficient dictionaries and noised distantly supervised instances at the pre-training stage. At the unifiedtraining stage, the two BERTs are trained together on human-annotated data to predict the correct labels for candidate regions. Empirical studies on three Chinese NER datasets demonstrate that our method achieves significant improvements against several baselines, establishing the new state-of-the-art performance.

show abstract

Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation

Cited by 11 publications

References 23 publications

Three Heads Are Better than One: Improving Cross-Domain NER with Progressive Decomposed Network

Three Heads Are Better than One: Improving Cross-Domain NER with Progressive Decomposed Network

Moving on from OntoNotes: Coreference Resolution Model Transfer

Toward Fully Exploiting Heterogeneous Corpus:A Decoupled Named Entity Recognition Model with Two-stage Training

Contact Info

Product

Resources

About