A Neural Local Coherence Model

Nguyen, Dat Tien; Joty, Shafiq

doi:10.18653/v1/p17-1121

Cited by 65 publications

(89 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Neural Grid (N&J) 3 . These are the neural versions of the entity grid models as proposed by (Tien Nguyen and Joty, 2017). They use convolutions over grammatical roles to model entity transitions in the distributed space.…”

Section: Models Comparedmentioning

confidence: 99%

A Unified Neural Coherence Model

Moon¹,

Mohiuddin²,

Joty³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

Recently, neural approaches to coherence modeling have achieved state-of-the-art results in several evaluation tasks. However, we show that most of these models often fail on harder tasks with more realistic application scenarios. In particular, the existing models underperform on tasks that require the model to be sensitive to local contexts such as candidate ranking in conversational dialogue and in machine translation. In this paper, we propose a unified coherence model that incorporates sentence grammar, inter-sentence coherence relations, and global coherence patterns into a common neural framework. With extensive experiments on local and global discrimination tasks, we demonstrate that our proposed model outperforms existing models by a good margin, and establish a new state-of-the-art.

show abstract

Section: Models Comparedmentioning

confidence: 99%

A Unified Neural Coherence Model

Moon¹,

Mohiuddin²,

Joty³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

show abstract

“…To reduce model variance, we run the WSJ experiments 5 times with different random initializations and the GCDC ones 10 times (following Lai and Tetreault (2018)), and average the predicted scores of the ensembles for the final evaluation. For the WSJ data, we use the same train/dev splits as Tien Nguyen and Joty (2017), and for GCDC, we follow Lai and Tetreault (2018) and split the training data with a 9:1 ratio for tuning.…”

Section: Methodsmentioning

confidence: 99%

Multi-Task Learning for Coherence Modeling

Farag¹,

Yannakoudakis²

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

We address the task of assessing discourse coherence, an aspect of text quality that is essential for many NLP tasks, such as summarization and language assessment. We propose a hierarchical neural network trained in a multitask fashion that learns to predict a documentlevel coherence score (at the network's top layers) along with word-level grammatical roles (at the bottom layers), taking advantage of inductive transfer between the two tasks. We assess the extent to which our framework generalizes to different domains and prediction tasks, and demonstrate its effectiveness not only on standard binary evaluation coherence tasks, but also on real-world tasks involving the prediction of varying degrees of coherence, achieving a new state of the art.

show abstract

“…The first task, discrimination, is usually evaluated as accuracy of the model in ranking the original text higher than a permuted one (we use 20 permutations per document following previous work [11,14,30]). In order to better analyse our results, we add to this metric two widely used ranking metrics, i.d.…”

Section: Methodsmentioning

confidence: 99%

“…The typical tasks on which local coherence models are currently evaluated are: sentence ordering discrimination, where the system needs to rank original documents higher than randomly permuted ones, and insertion, introduced by [14], where the system has to rank the position of a sentence removed from a document. The state of the art for these tasks was recently achieved by [30], which uses the entity grid as input to a Convolutional Neural Network. Table 1: We report the count of Dialogue Act tags, the average number of tokens per turn, the average count of turns per dialogue and counts of our Training, test and Developments splits for our three datasets: SWBD, AMI and Oasis.…”

Section: The Dialogue Act Tags Are Directly Taken From the Swbd Damslmentioning

confidence: 99%

Coherence Models for Dialogue

2018

View full text Add to dashboard Cite

Coherence across multiple turns is a major challenge for stateof-the-art dialogue models. Arguably the most successful approach to automatically learning text coherence is the entity grid, which relies on modelling patterns of distribution of entities across multiple sentences of a text. Originally applied to the evaluation of automatic summaries and the news genre, among its many extensions, this model has also been successfully used to assess dialogue coherence. Nevertheless, both the original grid and its extensions do not model intents, a crucial aspect that has been studied widely in the literature in connection to dialogue structure. We propose to augment the original grid document representation for dialogue with the intentional structure of the conversation. Our models outperform the original grid representation on both text discrimination and insertion, the two main standard tasks for coherence assessment across three different dialogue datasets, confirming that intents play a key role in modelling dialogue coherence.

show abstract

A Neural Local Coherence Model

Cited by 65 publications

References 15 publications

A Unified Neural Coherence Model

A Unified Neural Coherence Model

Multi-Task Learning for Coherence Modeling

Coherence Models for Dialogue

Contact Info

Product

Resources

About