Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER

Abstract

Recent advances in prompt-based learning have shown impressive results on few-shot text classification tasks by using cloze-style language prompts. There have been attempts on prompt-based learning for NER which use manually designed templates to predict entity types. However, these two-step methods may suffer from error propagation (from entity span detection), need to prompt for all possible text spans which is costly, and neglect the interdependency when predicting labels for different spans in a sentence. In this paper, we present a simple demonstration-based learning method for NER, which augments the prompt (learning context) with a few task demonstrations. Such demonstrations help the model learn the task better under low-resource settings and allow for span detection and classification over all tokens jointly. Here, we explore entity-oriented demonstration which selects an appropriate entity example per each entity type, and instance-oriented demonstration which retrieves a similar instance example. Through extensive experiments, we find empirically that showing entity example per each entity type, along with its example sentence, can improve the performance both in indomain and cross-domain settings by 1-3 F1 score.

Span-Based Fine-Grained Entity-Relation Extraction via Sub-Prompts Combination

“…Schick et al [28] proposed an approach, PET, which consists of defining pairs of cloze question patterns and verbalizers that help leverage the knowledge contained within pre-trained language models for downstream tasks. It has achieved outstanding performance in widespread NLP tasks [29][30][31], especially in information extraction tasks [3,[32][33][34][35]. Cui et al [32] proposed a template-based method for NER, treating NER as a language model ranking problem in a sequence-to-sequence framework, where original sentences and statement templates filled by candidate named entity span are regarded as the source sequence and the target sequence, respectively.…”

“…Cui et al [32] proposed a template-based method for NER, treating NER as a language model ranking problem in a sequence-to-sequence framework, where original sentences and statement templates filled by candidate named entity span are regarded as the source sequence and the target sequence, respectively. Lee et al [33] proposed demonstration-based learning, a simple-yet-effective way to incorporate automatically constructed auxiliary supervision. Instead of reformatting the NER task into the cloze-style template, they augment the original input instances by appending automatically created task demonstrations.…”

Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge

“…This paradigm requires no parameter updates and can achieve excellent results with just a few examples from downstream tasks. Since the effect of ICL is strongly related to the choice of demonstration examples, recent studies have explored several effective example selection methods, e.g., similaritybased retrieval method (Liu et al, 2021;Rubin et al, 2021), validation set scores based selection (Lee et al, 2021), gradient-based method (Wang et al, 2023b). These results indicate that reasonable example selection can improve the performance of LLMs.…”