Improving the Inference of Implicit Discourse Relations via Classifying Explicit Discourse Connectives

Abstract

Discourse relation classification is an important component for automatic discourse parsing and natural language understanding. The performance bottleneck of a discourse parser comes from implicit discourse relations, whose discourse connectives are not overtly present. Explicit discourse connectives can potentially be exploited to collect more training data to collect more data and boost the performance. However, using them indiscriminately has been shown to hurt the performance because not all discourse connectives can be dropped arbitrarily. Based on this insight, we investigate the interaction between discourse connectives and the discourse relations and propose the criteria for selecting the discourse connectives that can be dropped independently of the context without changing the interpretation of the discourse. Extra training data collected only by the freely omissible connectives improve the performance of the system without additional features.

Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification

“…Later, to make a full use of unlabelled data, several studies performed multi-task or unsupervised learning methods (Lan et al, 2013;Braud and Denis, 2015;Fisher and Simmons, 2015;Rutherford and Xue, 2015).…”

Rhetorical relations markers in Russian RST Treebank

“…In (Rutherford A. et al, 2015) authors investigated the criteria for selecting the discourse connectives that can be omitted without changing the context. M. Taboada and D. Das (Taboada M., Das D, 2013) suggest an exhaustive investigation of discourse relation clues.…”

Learning better discourse representation for implicit discourse relation recognition via attention networks

“…Different data selection methods for implicit DRR can be classified into the following categories: instance typicality [Wang et al, 2012], multi-task learning [Lan et al, 2013], domain adaptation [Braud and Denis, 2014;Ji et al, 2015], semi-supervised learning [Hernault et al, 2010;Fisher and Simmons, 2015] and explicit discourse connective classification [Rutherford and Xue, 2015].…”

“…Yee et al [2014] show how this memory is organized and retrieved in brain. In order to explore semantic memory in neural networks, we borrow ideas from recently introduced memory networks [Weston et al, 2014;Sukhbaatar et al, 2015;Kumar et al, 2015] to organize semantic memory as a distributed matrix and use an attention model to retrieve this distributed memory. The adaptation and utilization of semantic memory into implicit DRR, to the best of our knowledge, has never been investigated before.…”

“…This semantic knowledge can be stored in an external memory for a discourse recognizer just like the semantic memory for humans. Inspired by the semantic memory in cognitive neuroscience [Yee et al, 2014] as well as memory network [Weston et al, 2014;Sukhbaatar et al, 2015;Kumar et al, 2015] and attentional mechanisms [Mnih et al, 2014;Bahdanau et al, 2014;, we propose a neural network with se-mantic memory for implicit DRR, which refers to SeMDER. The philosophy of SeMDER includes: (1) the external semantic memory should be distributed as this allows easy computation; (2) the semantic memory should be easily accessed and retrieved; and (3) the retrieved content should be integrated into the comprehension of meanings of discourse arguments and their relations.…”