Named Entity Recognition with Bidirectional LSTM-CNNs

Abstract

Named entity recognition is a challenging task that has traditionally required large amounts of knowledge in the form of feature engineering and lexicons to achieve high performance.In this paper, we present a novel neural network architecture that automatically detects word-and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering. We also propose a novel method of encoding partial lexicon matches in neural networks and compare it to existing approaches. Extensive evaluation shows that, given only tokenized text and publicly available word embeddings, our system is competitive on the CoNLL-2003 dataset and surpasses the previously reported state of the art performance on the OntoNotes 5.0 dataset by 2.13 F1 points. By using two lexicons constructed from publicly-available sources, we establish new state of the art performance with an F1 score of 91.62 on CoNLL-2003 and 86.28 on OntoNotes, surpassing systems that employ heavy feature engineering, proprietary lexicons, and rich entity linking information.

De-identification of clinical notes via recurrent neural network and conditional random field

“…Similar to [18, 19], we introduce distributed representations for features and extend BI-LSTM by adding a hidden layer after the LSTM layer, which concatenates the word representations generated by LSTM layer and the feature representations, as shown in Figure 3. The features used in BI-LSTM-FEA are: sentence information, section information, general NER information and dictionary features, which are the same as features mentioned in section “CRF-based Method”.…”

“…In our system, an ensemble classifier is deployed to combine the outputs of three individual machine learning-based subsystems, and a rule-based subsystem is used to identify some formulaic PHI instances. The three machine learning-based subsystems are a CRF-based system with a large number of hand-crafted features [12], a bidirectional LSTM-based system without any hand-crafted features [16, 17], and a variant of bidirectional LSTM-based system with a small quantity of hand-crafted features [18, 19]. Moreover, we also evaluate our system on the 2014 i2b2 challenge corpus and compare it with other state-of-the-art systems.…”

“…In recent years, recurrent neural network (RNN) [34] has been widely used to tackle various kinds of NLP tasks, such as machine translation [35], NER [16, 17, 18, 19, 36, 37], word sense disambiguation [38], syntax parsing [39, 40], etc., and has shown great potential. For NER, there have already been several popular neural architectures, such as Senna (Collobert et al, 2011) [37], Char-WNN (dos Santos et al, 2015) [36], LSTM-CRF (Huang et al, 2015 [19] and Lample et al, 2016 [17]), LSTM-CNNs-CRF (Ma et al, 2016) [16], and LSTM-CNNs (Chiu et al, 2016) [18].…”

“…For NER, there have already been several popular neural architectures, such as Senna (Collobert et al, 2011) [37], Char-WNN (dos Santos et al, 2015) [36], LSTM-CRF (Huang et al, 2015 [19] and Lample et al, 2016 [17]), LSTM-CNNs-CRF (Ma et al, 2016) [16], and LSTM-CNNs (Chiu et al, 2016) [18]. Collobert et al (2011) [37] proposed a feed-forward neural network model with capitalization and discrete suffix features.…”

“…Ma et al (2016) [16] used a bidirectional LSTM model with character-level word representations modeled by CNN. Chiu et al (2016) [18] used the similar architecture to Ma et al (2016) [16] and further introduced some features such as the character type, capitalization, and lexicon features. Instead of CNN for character-level word representations, Lample et al (2016) [17] adopted LSTM to model character-level word representations.…”