FLAT: Chinese NER Using Flat-Lattice Transformer

Li, Xiaonan; Yan, Hang; Qiu, Xipeng; Huang, Xuanjing

doi:10.18653/v1/2020.acl-main.611

Cited by 343 publications

(163 citation statements)

References 19 publications

Supporting

Mentioning

163

Contrasting

Order By: Relevance

“…Last but not least, the pre-trained language model BERT has been extensively exploited in the Natural Language Processing (NLP) community since its introduction (Devlin et al, 2019;Conneau and Lample, 2019). Owing to BERT's ability to extract contextualized information, BERT has been successfully utilized to enhance various tasks substantially, such as the aspect-based sentiment analysis task , summarization (Zhong et al, 2019), named entity recognition (Yan et al, 2019;Li et al, 2020) and Chinese dependency parsing . However, most works used BERT as an encoder, and less work uses BERT to do generation (Wang and Cho, 2019;Conneau and Lample, 2019).…”

Section: Related Workmentioning

confidence: 99%

BERT for Monolingual and Cross-Lingual Reverse Dictionary

Yan¹,

Li²,

Qiu³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

Self Cite

View full text Add to dashboard Cite

Reverse dictionary is the task to find the proper target word given the word description. In this paper, we tried to incorporate BERT into this task. However, since BERT is based on the byte-pair-encoding (BPE) subword encoding, it is nontrivial to make BERT generate a word given the description. We propose a simple but effective method to make BERT generate the target word for this specific task. Besides, the cross-lingual reverse dictionary is the task to find the proper target word described in another language. Previous models have to keep two different word embeddings and learn to align these embeddings. Nevertheless, by using the Multilingual BERT (mBERT), we can efficiently conduct the crosslingual reverse dictionary with one subword embedding, and the alignment between languages is not necessary. More importantly, mBERT can achieve remarkable cross-lingual reverse dictionary performance even without the parallel corpus, which means it can conduct the cross-lingual reverse dictionary with only corresponding monolingual data. Code is publicly available at https://github.com/ yhcc/BertForRD.git.

show abstract

Section: Related Workmentioning

confidence: 99%

BERT for Monolingual and Cross-Lingual Reverse Dictionary

Yan¹,

Li²,

Qiu³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…It uses the existing neural network model to model the input sequence. The main neural network models are network models based on RNN and its variants [42,43], models based on CNN [44,45], and models based on Transformer [46,47].…”

Section: Model Frameworkmentioning

confidence: 99%

“…The network model based on Transformer. Li Xiaonan et al [46] proposed a FLAT(Flat-LAttice Transformer) structure to be applied to Chinese NER. This model relies on the powerful functions of Transformer and carefully designed specific location codes to fully utilize lattice information and has efficient parallelism.…”

Section: Model Frameworkmentioning

confidence: 99%

A review of Chinese named entity recognition

Cheng

Liu

et al. 2021

KSII TIIS

View full text Add to dashboard Cite

Named Entity Recognition (NER) is used to identify entity nouns in the corpus such as Location, Person and Organization, etc. NER is also an important basic of research in various natural language fields. The processing of Chinese NER has some unique difficulties, for example, there is no obvious segmentation boundary between each Chinese character in a Chinese sentence. The Chinese NER task is often combined with Chinese word segmentation, and so on. In response to these problems, we summarize the recognition methods of Chinese NER. In this review, we first introduce the sequence labeling system and evaluation metrics of NER. Then, we divide Chinese NER methods into rule-based methods, statistics-based machine learning methods and deep learning-based methods. Subsequently, we analyze in detail the model framework based on deep learning and the typical Chinese NER methods. Finally, we put forward the current challenges and future research directions of Chinese NER technology.

show abstract

“…A high quality of text representation plays an important role to obtain good performance for many NLP tasks (Song et al, 2017;Zhu et al, 2019;Liu and Lapata, 2019), where a powerful encoder is required to model more contextual information. Inspired by the studies (Song et al, 2009;Song and Xia, 2012;Ouyang et al, 2017;Kim et al, 2018;Peng et al, 2018;Higashiyama et al, 2019;Tian et al, 2020c;Li et al, 2020) that leverage the large granularity contextual information carried by n-grams to enhance text representation for Chinese, we propose ZEN to enhance character based text encoders (e.g., BERT) by leveraging ngrams. In doing so, we extract n-grams prior to pre-training ZEN through two different steps.…”

Section: N-gram Extractionmentioning

confidence: 99%

ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations

Diao¹,

Bai

Yan

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

The pre-training of text encoders normally processes text as a sequence of tokens corresponding to small text units, such as word pieces in English and characters in Chinese. It omits information carried by larger text granularity, and thus the encoders cannot easily adapt to certain combinations of characters. This leads to a loss of important semantic information, which is especially problematic for Chinese because the language does not have explicit word boundaries. In this paper, we propose ZEN, a BERT-based Chinese (Z) text encoder Enhanced by N-gram representations, where different combinations of characters are considered during training, thus potential word or phrase boundaries are explicitly pre-trained and fine-tuned with the character encoder (BERT). Therefore ZEN incorporates the comprehensive information of both the character sequence and words or phrases it contains. Experimental results illustrated the effectiveness of ZEN on a series of Chinese NLP tasks, where state-of-the-art results is achieved on most tasks with requiring less resource than other published encoders. It is also shown that reasonable performance is obtained when ZEN is trained on a small corpus, which is important for applying pre-training techniques to scenarios with limited data. 1 * Work done during the internship at Sinovation Ventures.

show abstract

FLAT: Chinese NER Using Flat-Lattice Transformer

Cited by 343 publications

References 19 publications

BERT for Monolingual and Cross-Lingual Reverse Dictionary

BERT for Monolingual and Cross-Lingual Reverse Dictionary

A review of Chinese named entity recognition

ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations

Contact Info

Product

Resources

About