Zero-Resource Cross-Domain Named Entity Recognition

Abstract

Cross-domain named entity recognition (NER) models are able to cope with the scarcity issue of NER samples in target domains. However, most of the existing NER benchmarks lack domain-specialized entity types or do not focus on a certain domain, leading to a less effective cross-domain evaluation. To address these obstacles, we introduce a cross-domain NER dataset (CrossNER), a fully-labeled collection of NER data spanning over five diverse domains with specialized entity categories for different domains. Additionally, we also provide a domain-related corpus since using it to continue pre-training language models (domain-adaptive pre-training) is effective for the domain adaptation. We then conduct comprehensive experiments to explore the effectiveness of leveraging different levels of the domain corpus and pre-training strategies to do domain-adaptive pre-training for the crossdomain task. Results show that focusing on the fractional corpus containing domain-specialized entities and utilizing a more challenging pre-training strategy in domain-adaptive pre-training are beneficial for the NER domain adaptation, and our proposed method can consistently outperform existing cross-domain NER baselines. Nevertheless, experiments also illustrate the challenge of this cross-domain NER task. We hope that our dataset and baselines will catalyze research in the NER domain adaptation area. The code and data are available at https://github.com/zliucr/CrossNER.

GEMNET: Effective Gated Gazetteer Representations for Recognizing Complex Entities in Low-context Input

“…It has demonstrated to be useful in various applications like recommendation (Zhu et al, 2020), domain adaptation for sentiment analysis, and POS tagging (Guo et al, 2018). For NER, Liu et al (2020) proposed a Mixture of Entity Experts (MoEE) approach where they train an expert layer for each entity type, and then combine them using an MoE approach. Their approach does not include external gazetteers, and the experts provide an independent representation that is not combined with the word representation.…”

Model Generalization on COVID-19 Fake News Detection

“…In general, recent works on model generalizability can be divided into two different directions: 1) adaptive training and 2) robust loss function. In adaptive training, different meta-learning [5] and fast adaptation [18,20,35] approaches have been developed and show promising result for improving the generalization of the model over different domains. Another meta-learning approach, called meta transfer learning [34], improves the generalization ability for a low-resource domain by leveraging a high-resource domain dataset.…”

Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack

“…For a fair comparison, we adopt the same network architecture BiLSTM (Hochreiter and Schmidhuber, 1997) as previous work (Bapna et al, 2017;Lee and Jha, 2019;Shah et al, 2019;Liu et al, 2020a;Liu et al, 2020b). Given an utterance with n tokens as w = [w 1 , w 2 , .…”

“…The main drawback is the multiple prediction problem where a word may be predicted as multiple slot types. In contrast, (Liu et al, 2020a;Liu et al, 2020b) propose a two-stage slot filling framework. They first predict whether the tokens are slot entities or not by a BIO 3-way classifier, then identify their specific slot types based on slot type descriptions.…”