NNE: A Dataset for Nested Named Entity Recognition in English Newswire

Abstract

Named entity recognition (NER) is widely used in natural language processing applications and downstream tasks. However, most NER tools target flat annotation from popular datasets, eschewing the semantic information available in nested entity mentions. We describe NNE-a fine-grained, nested named entity dataset over the full Wall Street Journal portion of the Penn Treebank (PTB). Our annotation comprises 279,795 mentions of 114 entity types with up to 6 layers of nesting. We hope the public release of this large dataset for English newswire will encourage development of new techniques for nested NER.

A Bidirectional Iterative Algorithm for Nested Named Entity Recognition

“…Our experiments were conducted on four nested named entity recognition datasets: GENIA [5] (biomedical domain), NNE [2] (news domain), PolEval [33] (mixed texts, a Polish corpus), and GermEval [34] (news and Wikipedia, a German corpus). Each dataset was split into three parts: training, validation, and test sets.…”

“…It contains annotations of the Wall Street Journal portion of the Penn Treebank (PTB). The total number of named entities is 279 795, of which 118 525 [2] evaluated three variants of baseline model based on BiLSTM-CRF architecture: (a) a flat NER model detecting only the outermost entities; (b) a flat NER model detecting only the innermost entities; and (c) a combination of predictions of those two models. Our comparison also includes other known methods, retrained and evaluated using publicly available source codes.…”

“…Our comparison also includes other known methods, retrained and evaluated using publicly available source codes. † denotes methods evaluated by Ringland et al [2], ‡ denotes methods which we evaluated. are outer entities.…”

“…The NNE dataset was introduced recently in a study by Ringland et al [2] which also included the evaluation of three baseline models, in addition to a transition-based model by Wang et al [22], and a hypergraph based model by Wang and Lu [15]. We compare our algorithm with these approaches in Table 4.…”

“…In practice, however, named entities frequently contain names of other entities, and many applications could benefit from the ability to identify hierarchical structures of mentions. In Figure 1, we demonstrate an example of a sentence from the NNE [2] corpus annotated with a set of overlapping tags.…”