End-to-End Neural Context Reconstruction in Chinese Dialogue

Abstract

We tackle the problem of context reconstruction in Chinese dialogue, where the task is to replace pronouns, zero pronouns, and other referring expressions with their referent nouns so that sentences can be processed in isolation without context. Following a standard decomposition of the context reconstruction task into referring expression detection and coreference resolution, we propose a novel end-toend architecture for separately and jointly accomplishing this task. Key features of this model include POS and position encoding using CNNs and a novel pronoun masking mechanism. One perennial problem in building such models is the paucity of training data, which we address by augmenting previouslyproposed methods to generate a large amount of realistic training data. The combination of more data and better models yields accuracy higher than the state-of-the-art method in coreference resolution and end-to-end context reconstruction.

Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation

“…Open domain question answering systems usually follow a two-step approach: first retrieve question relevant passages, and then scan the returned text to identify the answer span using a reading comprehension model (Jurafsky and Martin, 2018;Kratzwald and Feuerriegel, 2018;Yang et al, 2019a). Prior work has focused on the answer span annotation task and has even achieved super human performance on some datasets.…”

Tackling Zero Pronoun Resolution and Non-Zero Coreference Resolution Jointly

“…These works rely on handcraft features and suffer from error propagation. To solve these problems, Yang et al (2019) tackles context reconstruction in an end-to-end way. It formulates pronouns detection as a sequence labeling task and uses pronoun masking mechanism to combine the detection and the resolution modules.…”

“…However, positions of zero pronouns are usually unknown in practice. To solve this problem, some recent works (Yang et al, 2019;Song et al, 2020) treat spaces between tokens as candidate zero pronouns, recognize and resolve zero pronouns based on gap embeddings jointly. For non-zero coreference resolution, the state-of-the-art models belong to an end-to-end paradigm or its variants.…”

Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation

“…Open domain question answering is the problem of answering a question from a large collection of documents (Voorhees and Tice, 2000;Chen et al, 2017). Systems usually follow a two-step approach: first retrieve question relevant passages, and then scan the returned text to identify the answer span using a reading comprehension model (Jurafsky and Martin, 2018;Kratzwald and Feuerriegel, 2018;Yang et al, 2019a). Prior work has focused on the answer span annotation task and has even achieved super human performance on some datasets.…”