An Investigation of Potential Function Designs for Neural CRF

Hu, Zechuan; Jiang, Yong; Bach, Nguyen; Wang, Tao; Huang, Zhongqiang; Huang, Fei; Tu, Kewei

doi:10.18653/v1/2020.findings-emnlp.236

Cited by 3 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BCRF and support examples have shown effective on all 8 settings. We further confirmed the performance of BCRF without the label encoder by using randomly initialized label embeddings as in Hu et al (2020) (-SupEx, +BCRF[R]). The low performance of this setting indicates the strength of BCRF combined with the label encoder.…”

Section: The Effects Of Support Examples and Bcrfmentioning

confidence: 57%

See 1 more Smart Citation

Few-shot Named Entity Recognition with Supported and Dependent Label Representations

Miura,

Takahashi

2023

Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings)

View full text Add to dashboard Cite

We explore the problem of few-shot named entity recognition (NER) by introducing two ideas to improve label representations. Recently, the use of token representations with a distance metric has been shown to be effective in few-shot NER, and we take an approach to use label representations along with token representations. Firstly, we add support examples to a label name (e.g., "person; example: Federic Krupp, Gao, Honecker, Bush, Deverow") when obtaining a label representation. Secondly, we estimate a transition score among labels with a bilinear function among label representations. The proposed approach is evaluated on 4 open few-shot NER datasets and we found that the approach can improve the performance of one-stage few-shot NER.

show abstract

Section: The Effects Of Support Examples and Bcrfmentioning

confidence: 57%

“…The estimation of a transition score has been investigated in a more resource rich setting in Hu et al (2020). Our BCRF score takes a simple estimation approach since we focus on a resource poor few-shot setting.…”

Section: Estimation Of Transition Scorementioning

confidence: 99%

Few-shot Named Entity Recognition with Supported and Dependent Label Representations

Miura,

Takahashi

2023

Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings)

View full text Add to dashboard Cite

show abstract

“…Structured prediction is the task of mapping input sentences to structured outputs. It is a fundamental task in natural language processing and has many applications, i.e., sequence labeling (DeRose, 1988;Lample et al, 2016;Ma and Hovy, 2016;Hu et al, 2020b), dependency parsing (Chen and Manning, 2014;Dozat and Manning, 2016;Ahmad et al, 2019) and semantic role labeling (van der Plas et al, 2011;Strubell et al, 2018;Cai and Lapata, 2020).…”

Section: Introductionmentioning

confidence: 99%

Multi-View Cross-Lingual Structured Prediction with Minimum Supervision

Hu¹,

Jiang²,

Bach³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

In structured prediction problems, crosslingual transfer learning is an efficient way to train quality models for low-resource languages, and further improvement can be obtained by learning from multiple source languages. However, not all source models are created equal and some may hurt performance on the target language. Previous work has explored the similarity between source and target sentences as an approximate measure of strength for different source models. In this paper, we propose a multi-view framework, by leveraging a small number of labeled target sentences, to effectively combine multiple source models into an aggregated source view at different granularity levels (language, sentence, or sub-structure), and transfer it to a target view based on a task-specific model. By encouraging the two views to interact with each other, our framework can dynamically adjust the confidence level of each source model and improve the performance of both views during training. Experiments for three structured prediction tasks on sixteen data sets show that our framework achieves significant improvement over all existing approaches, including these with access to additional source language data.

show abstract

“…Sequence labeling is an important task in natural language processing. It has many applications such as Part-of-Speech Tagging (POS) (DeRose, 1988;Toutanova et al, 2003) and Named Entity Recognition (NER) (Ratinov and Roth, 2009;Ritter et al, 2011;Lample et al, 2016;Ma and Hovy, 2016;Hu et al, 2020). Approaches to sequence labeling are mostly based on supervised learning, which relies heavily on labeled data.…”

Section: Introductionmentioning

confidence: 99%

Risk Minimization for Zero-shot Sequence Labeling

Hu¹,

Jiang²,

Bach³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

Zero-shot sequence labeling aims to build a sequence labeler without human-annotated datasets. One straightforward approach is utilizing existing systems (source models) to generate pseudo-labeled datasets and train a target sequence labeler accordingly. However, due to the gap between the source and the target languages/domains, this approach may fail to recover the true labels. In this paper, we propose a novel unified framework for zero-shot sequence labeling with minimum risk training and design a new decomposable risk function that models the relations between the predicted labels from the source models and the true labels. By making the risk function trainable, we draw a connection between minimum risk training and latent variable model learning. We propose a unified learning algorithm based on the expectation maximization (EM) algorithm. We extensively evaluate our proposed approaches on cross-lingual/domain sequence labeling tasks over twenty-one datasets. The results show that our approaches outperform state-of-the-art baseline systems.

show abstract

An Investigation of Potential Function Designs for Neural CRF

Cited by 3 publications

References 18 publications

Few-shot Named Entity Recognition with Supported and Dependent Label Representations

Few-shot Named Entity Recognition with Supported and Dependent Label Representations

Multi-View Cross-Lingual Structured Prediction with Minimum Supervision

Risk Minimization for Zero-shot Sequence Labeling

Contact Info

Product

Resources

About