Spectral Semi-Supervised Discourse Relation Classification

Abstract

Discourse parsing is the process of discovering the latent relational structure of a long form piece of text and remains a significant open challenge. One of the most difficult tasks in discourse parsing is the classification of implicit discourse relations. Most state-of-the-art systems do not leverage the great volume of unlabeled text available on the web-they rely instead on human annotated training data. By incorporating a mixture of labeled and unlabeled data, we are able to improve relation classification accuracy, reduce the need for annotated data, while still retaining the capacity to use labeled data to ensure that specific desired relations are learned. We achieve this using a latent variable model that is trained in a reduced dimensionality subspace using spectral methods. Our approach achieves an F 1 score of 0.485 on the implicit relation labeling task for the Penn Discourse Treebank.

Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification

“…With the release of PDTB 2.0, a number of studies performed discourse relation recognition on natural (i.e., genuine) discourse data with the use of traditional NLP techniques to extract linguistically informed features and traditional machine learning algorithms (Pitler et al, 2009;Lin et al, 2009;Wang et al, 2010;Braud and Denis, 2015;Fisher and Simmons, 2015).…”

“…Later, to make a full use of unlabelled data, several studies performed multi-task or unsupervised learning methods (Lan et al, 2013;Braud and Denis, 2015;Fisher and Simmons, 2015;Rutherford and Xue, 2015).…”

“…With the release of PDTB 2.0 (Prasad et al, 2008), lots of work has been done for discourse relation identification on natural (i.e., genuine) discourse data (Pitler et al, 2009;Lin et al, 2009;Wang et al, 2010;Zhou et al, 2010;Braud and Denis, 2015;Fisher and Simmons, 2015) with the use of traditional NLP linguistically informed features and machine learning algorithms. Recently, more and more researchers resorted to neural networks for implicit discourse recognition (Zhang et al, 2015;Qin et al, 2016a;Liu and Li, 2016;Braud and Denis, 2016;Wu et al, 2016).…”

Proceedings of the CoNLL-16 shared task

“…It can be of great benefit to many downstream natural language processing (NLP) applications and has attracted lots of research Lan et al, 2013;Lin et al, 2014;Fisher and Simmons, 2015;. Following the first edition in CoNLL-2015 , CoNLL-2016 is the 2nd edition of the CoNLL Shared Task on Shallow Discourse Parsing, which contains following tasks: discourse parsing task and supplementary task (sense classification using gold standard argument pairs) in English and Chinese.…”

“…We can see that the named efficient features such as lexical and syntactic features (word co-occurrences, function words, phrase or dependency parses), partial shallow semantic features (co-reference patterns, semantic attribute of words, e.g., polarity) and a few dynamic features are adopted in existing works Zhou et al, 2010;Prasad et al, 2010;Feng and Hirst, 2012;. In response to the data scarcity problem, semi-supervised and unsupervised methods are explored for implicit relations inference in recent years (Hernault et al, 2011;Hong et al, 2012;Lan et al, 2013;Fisher and Simmons, 2015). Experiments demonstrate that these kinds of methods can acquire more stable statistical distribution via large scale unlabeled corpus hence achieve higher classification accuracy.…”