An Encoding Strategy Based Word-Character

Abstract

A recently proposed lattice model has demonstrated that words in character sequence can provide rich word boundary information for character-based Chinese NER model. In this model, word information is integrated into a shortcut path between the start and the end characters of the word. However, the existence of shortcut path may cause the model to degenerate into a partial word-based model, which will suffer from word segmentation errors. Furthermore, the lattice model can not be trained in batches due to its DAG structure. In this paper, we propose a novel wordcharacter LSTM(WC-LSTM) model to add word information into the start or the end character of the word, alleviating the influence of word segmentation errors while obtaining the word boundary information. Four different strategies are explored in our model to encode word information into a fixed-sized representation for efficient batch training. Experiments on benchmark datasets show that our proposed model outperforms other stateof-the-arts models.

ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations

“…Overall, in both R and P settings, ZEN outperforms BERT in all seven tasks, which clearly indicates the advantage of introducing n-grams into the encoding of character sequences. 13 This observation is similar to that from Dos Santos and Gatti (2014); Lample et al (2016); Bojanowski et al (2017); Liu et al (2019a). In detail, when compare R and P settings, 12 Most of the previous studies show their performance on the development set of the aforementioned tasks and we follow them to do so in order to provide a reference and comparison.…”

Overview of CCKS 2020 Task 3: Named Entity Recognition and Event Extraction in Chinese Electronic Medical Records

“…A drawback of the purely character-based NER model is that the word information is not fully exploited. To incorporate word information in Chinese NER, some recent methods, such as [10,11,12,13,14], resort to an automatically constructed lexicon.…”

Incorporating Uncertain Segmentation Information into Chinese NER for Social Media Text

“…Chinese word segmentation was performed first before applying character sequence labeling (Guo et al, 2004;Mao et al, 2008;Zhu and Wang, 2019). The pre-processing segmentation features included character positional embedding (Peng and Dredze, 2015;He and Sun, 2017a,b), segmentation tags Zhu and Wang, 2019), word embedding (Peng and Dredze, 2015;Liu et al, 2019;E and Xiang, 2017) and so on. The other was to train NER and CWS tasks jointly to incorporate task-shared word boundary information from the CWS into the NER (Xu et al, 2013;Peng and Dredze, 2016;Cao et al, 2018).…”

“…However, they treated the segmentations equally without error discrimination. Liu et al (2019) introduced four naive selection strategies to select words from the pre-prepared Lexicon for their model. However, these strategies did not consider the context of a sentence.…”

“…proposed a Lattice LSTM model that used the gated recurrent units to control the contribution of the potential words. However, as shown by Liu et al (2019), the gate mechanism might cause the model to degenerate into a partial word-based model. Ding et al (2019) and Gui et al (2019) proposed the models with graph neural network based on the information that the gazetteers or lexicons offered.…”