Decomposed Meta-Learning for Few-Shot Named Entity Recognition

Abstract

Few-shot named entity recognition (NER) systems aim at recognizing novel-class named entities based on only a few labeled examples. In this paper, we present a decomposed metalearning approach which addresses the problem of few-shot NER by sequentially tackling fewshot span detection and few-shot entity typing using meta-learning. In particular, we take the few-shot span detection as a sequence labeling problem and train the span detector by introducing the model-agnostic meta-learning (MAML) algorithm to find a good model parameter initialization that could fast adapt to new entity classes. For few-shot entity typing, we propose MAML-ProtoNet, i.e., MAML-enhanced prototypical networks to find a good embedding space that can better distinguish text span representations from different entity classes. Extensive experiments on various benchmarks show that our approach achieves superior performance over prior methods. 1 * Equal contributions.

SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition

“…prototypical learning (Snell et al, 2017), margin-based learning (Levi et al, 2021) and contrastive learning . Existing approaches can be divided into two kinds, i.e., onestage (Snell et al, 2017;Hou et al, 2020;Das et al, 2022;Ziyadi et al, 2020) and two-stage (Ma et al, 2022b;Wu et al, 2022;Shen et al, 2021). Generally, the methods in the kind of one-stage typically categorize the entity type by the token-level metric learning.…”

“…In contrast, two-stage mainly focuses on two training stages consist of entity span extraction and mention type classification. (Ma et al, 2022b) is a related work of our paper, which utilizes model-agnostic meta-learning (MAML) (Finn et al, 2017) to improve the adaptation ability of the two-stage model. Different from them, we aim to 1) further improve the performance on detecting and extracting the candidate entity spans, and 2) alleviate the bottleneck of false positives in the two-stage few-shot NER system.…”

“…Yet, these conventional approaches heavily depend on the time-consuming and laborintensive process of data annotation. Thus, an attractive problem of few-shot NER (Ding et al, 2021;Huang et al, 2021;Ma et al, 2022b) has been introduced which involves recognizing novelclass entities based on very few labeled examples (i.e. support examples).…”

“…"BIO" rules) Shen et al, 2021). To bypass these issues, a branch of two-stage methods arise to decompose NER into two separate processes (Shen et al, 2021;Ma et al, 2022b;Wu et al, 2022), including span extraction and mention classification. Specifically, they first extract multiple spans via a class-agnostic model and then assign the label for each predicted span based on metric learning.…”

“…For span extraction, we introduce a Span Extractor, which aims to find the candidate spans. Different from the recent work (Ma et al, 2022b) which models it as a sequence labeling task, we convert the sequential tags to a global boundary matrix, which denotes the sentence-level target label, enabling the model to learn the explicit span boundary information regardless of the token-wise label dependency. For mention classification, we propose a Mention Classifier which aims at assigning a pre-defined entity type for each recalled span.…”