Neural recovery machine for Chinese dropped pronoun

Abstract

Dropped pronouns (DPs) are ubiquitous in pro-drop languages like Chinese, Japanese etc. Previous work mainly focused on painstakingly exploring the empirical features for DPs recovery. In this paper, we propose a neural recovery machine (NRM) to model and recover DPs in Chinese, so that to avoid the non-trivial feature engineering process. The experimental results show that the proposed NRM significantly outperforms the state-of-the-art approaches on both two heterogeneous datasets. Further experiment results of Chinese zero pronoun (ZP) resolution show that the performance of ZP resolution can also be improved by recovering the ZPs to DPs.

Evaluating the impact of citations of articles based on knowledge flow patterns hidden in the citations

“…Deep learning has been successful in several fields because of the strong ability of feature learning and modeling [48–49]. The use of distributed representation in deep learning has shown high effectiveness in capturing the semantics of words, phrases, and sentences, which benefits natural language–processing applications such as sentiment analysis [50], syntactic parsing [51–54], text summarization [55], and others [56–59]. This research has explored the use of distributed document representation in calculating the content similarity between articles.…”

Transformer-GCRF: Recovering Chinese Dropped Pronouns with General Conditional Random Fields

“…Giannella et al (2017) employed a linear-chain CRF to jointly predict the position, person, and number of the dropped pronouns in a single utterance, to exploit the sequential nature of this problem. With the powerful representation capability of neural network (Xu et al, 2020), Zhang et al (2016) introduced a MLP neural network to recover the dropped pronouns based on the concatenation of word embeddings within a fixed-length window. proposed a neural network with structured attention to model the interaction between dropped pronouns and their referents using both sentence-level and word-level context, and again each dropped pronoun is predicted independently.…”

“…We used the TC section which consists of transcripts of Chinese telephone conversation speech. The BaiduZhidao dataset is a question answering dialogue corpus collected by (Zhang et al, 2016). Ten types of dropped pronouns are annotated according to the pronoun annotation guidelines.…”

“…Baselines: State-of-the-art dropped pronoun recovery models are used as baselines: (1) MEPR (Yang et al, 2015) which leverages a set of elaborately designed features and trains a Maximum Entropy classifier to predict the type of dropped pronoun before each token; (2) NRM (Zhang et al, 2016) which employs two separate MLPs to predict the position and type of a dropped pronoun utilizing representation of words in a fixed-length window; (3) BiGRU which utilizes a bidirectional RNN to encode each token in a pro-drop sentence and makes prediction based on the encoded states; (4) NDPR which models dropped pronoun referents by attending to the context and independently predicts the presence and type of DP for each token.…”

“…For example, Yang et al (2015) first studied this problem in SMS data and utilized a Maximum Entropy classifier to recover dropped pronouns. Deep neural networks such as Multi-Layer Perceptrons (MLPs) and structured attention networks have also been used to tackle this problem (Zhang et al, 2016;. Giannella et al (2017) used a linear-chain CRF to model the dependency be-tween the sequence of predictions in a utterance.…”