The CoNLL-2015 Shared Task on Shallow Discourse Parsing

Abstract

The CoNLL-2015 Shared Task is on Shallow Discourse Parsing, a task focusing on identifying individual discourse relations that are present in a natural language text. A discourse relation can be expressed explicitly or implicitly, and takes two arguments realized as sentences, clauses, or in some rare cases, phrases. Sixteen teams from three continents participated in this task. For the first time in the history of the CoNLL shared tasks, participating teams, instead of running their systems on the test set and submitting the output, were asked to deploy their systems on a remote virtual machine and use a web-based evaluation platform to run their systems on the test set. This meant they were unable to actually see the data set, thus preserving its integrity and ensuring its replicability. In this paper, we present the task definition, the training and test sets, and the evaluation protocol and metric used during this shared task. We also summarize the different approaches adopted by the participating teams, and present the evaluation results. The evaluation data sets and the scorer will serve as a benchmark for future research on shallow discourse parsing.

OPT: Oslo–Potsdam–Teesside. Pipelining Rules, Rankers, and Classifier Ensembles for Shallow Discourse Parsing

“…To compare against the previous state of the art, we include results for the top-performing systems from the 2015 and 2016 competitions (as reported by Xue et al, 2015, andXue et al, 2016, respectively). Where applicable, best results (when comparing F 1 ) are highlighted for each sub-task and -metric.…”

Robust Non-Explicit Neural Discourse Parser in English and Chinese

“…Word vectors English word vectors are taken from 300-dimensional Skip-gram word vectors trained on Google News data, provided by the shared task organizers (Mikolov et al, 2013;Xue et al, 2015). We trained our own 250-dimensional Chinese word vectors on Gigaword corpus, which is the same corpus used by the 300-dimensional Chinese word vectors provided by the shared task organizers (Graff and Chen, 2005).…”

Towards Full Text Shallow Discourse Relation Annotation: Experiments with Cross-Paragraph Implicit Relations in the PDTB

“…For example, Pitler et al (2009) report improvements in implicit relation sense classification with a sequence model. And more re-cent systems, including the best systems (Wang and Lan, 2015;Oepen et al, 2016) at the recent CONLL shared tasks on PDTB-style shallow discourse parsing (Xue et al, 2015(Xue et al, , 2016, while not using a sequence model, still incorporate features about neighboring relations. Such systems have many applications, including summarization (Louis et al, 2010), information extraction (Huang and Riloff, 2012), question answering (Blair-Goldensohn, 2007), opinion analysis (Somasundaran et al, 2008), and argumentation (Zhang et al, 2016).…”