Coherence Models for Dialogue

Cervone, Alessandra; Stepanov, Evgeny A.; Riccardi, Giuseppe

doi:10.21437/interspeech.2018-2446

Cited by 23 publications

(44 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…EntityGrid and its extensions (Burstein et al, 2010;Guinaudeau and Strube, 2013;Mesgar and Strube, 2014;Tien Nguyen and Joty, 2017;Farag and Yannakoudakis, 2019) rely on entity transitions, as proxies of semantic connectivity, between utterances. These approaches are agnostic to discourse properties of dialogues (Purandare and Litman, 2008;Gandhe and Traum, 2008;Cervone et al, 2018 Inspired by EntityGrid, Gandhe and Traum (2016) define transition patterns among DA labels associated with utterances to measure coherence. Cervone et al (2018) combine the above ideas by augmenting entity grids with utterance DA labels.…”

Section: Related Workmentioning

confidence: 99%

“…dialogue from a random sequence of dialogue utterances (Halliday and Hasan, 1976;Grosz and Sidner, 1986;Byron and Stent, 1998). Dialogue coherence deals with semantic relations between utterances considering their dialogue acts (Perrault and Allen, 1978;Cervone et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

“…We release our source code 1 .Recent approaches to dialogue coherence modeling use the coherence features designed for monologue texts, e.g. entity transitions (Barzilay and Lapata, 2005), and augment them with dialoguerelevant features, e.g., DA labels (Cervone et al, 2018). These DA labels are provided by human annotators or DA prediction models.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Dialogue Coherence Assessment Without Explicit Dialogue Act Labels

Mesgar

Bücker

Gurevych

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Recent dialogue coherence models use the coherence features designed for monologue texts, e.g. nominal entities, to represent utterances and then explicitly augment them with dialogue-relevant features, e.g., dialogue act labels. It indicates two drawbacks, (a) semantics of utterances is limited to entity mentions, and (b) the performance of coherence models strongly relies on the quality of the input dialogue act labels. We address these issues by introducing a novel approach to dialogue coherence assessment. We use dialogue act prediction as an auxiliary task in a multi-task learning scenario to obtain informative utterance representations for coherence assessment. Our approach alleviates the need for explicit dialogue act labels during evaluation. The results of our experiments show that our model substantially (more than 20 accuracy points) outperforms its strong competitors on the Dai-lyDialogue corpus, and performs on par with them on the SwitchBoard corpus for ranking dialogues concerning their coherence. We release our source code 1 .Recent approaches to dialogue coherence modeling use the coherence features designed for monologue texts, e.g. entity transitions (Barzilay and Lapata, 2005), and augment them with dialoguerelevant features, e.g., DA labels (Cervone et al., 2018). These DA labels are provided by human annotators or DA prediction models. Such coherence models suffer from the following drawbacks: (a) they curb semantic representations of utterances to entities, which are sparse in dialogue because of short utterance lengths, and (b) their performance relies on the quality of their input DA labels.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dialogue Coherence Assessment Without Explicit Dialogue Act Labels

Mesgar

Bücker

Gurevych

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…• Entity Grid: Cervone et al (2018); Barzilay and Lapata (2008) show that entities and DA transitions across turns can be strong features for assessing dialog coherence. Starting from a grid representation of the turns of the conversation as a matrix (DAs × entities), these features are designed to capture the patterns of topic and intent shift distribution of a dialog.…”

Section: Featuresmentioning

confidence: 99%

Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators

Yi¹,

Goel²,

Khatri³

et al. 2019

Proceedings of the 12th International Conference on Natural Language Generation

Self Cite

View full text Add to dashboard Cite

Encoder-decoder based neural architectures serve as the basis of state-of-the-art approaches in end-to-end open domain dialog systems. Since most of such systems are trained with a maximum likelihood (MLE) objective they suffer from issues such as lack of generalizability and the generic response problem, i.e., a system response that can be an answer to a large number of user utterances, e.g., "Maybe, I don't know." Having explicit feedback on the relevance and interestingness of a system response at each turn can be a useful signal for mitigating such issues and improving system quality by selecting responses from different approaches. Towards this goal, we present a system that evaluates chatbot responses at each dialog turn for coherence and engagement. Our system provides explicit turn-level dialog quality feedback, which we show to be highly correlated with human evaluation. To show that incorporating this feedback in the neural response generation models improves dialog quality, we present two different and complementary mechanisms to incorporate explicit feedback into a neural response generation model: reranking and direct modification of the loss function during training. Our studies show that a response generation model that incorporates these combined feedback mechanisms produce more engaging and coherent responses in an open-domain spoken dialog setting, significantly improving the response quality using both automatic and human evaluation.

show abstract

“…Then it predicts the most likely DA of the next response based on the outputs of RNNs. Cervone et al (2018) showed that a DA is useful to improve the coherency of response. The predicted DAs can be used to generate a future response, which adds controllability and interpretability into a neural di-alogue system.…”

Section: Introductionmentioning

confidence: 99%

Dialogue-Act Prediction of Future Responses Based on Conversation History

Tanaka¹,

Takayama²,

Arase³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

View full text Add to dashboard Cite

Sequence-to-sequence models are a common approach to develop a chatbot. They can train a conversational model in an end-to-end manner. One significant drawback of such a neural network based approach is that the response generation process is a black-box, and how a specific response is generated is unclear. To tackle this problem, an interpretable response generation mechanism is desired. As a step toward this direction, we focus on dialogue-acts (DAs) that may provide insight to understand the response generation process. In particular, we propose a method to predict a DA of the next response based on the history of previous utterances and their DAs. Experiments using a Switch Board Dialogue Act corpus show that compared to the baseline considering only a single utterance, our model achieves 10.8% higher F1-score and 3.0% higher accuracy on DA prediction.

show abstract

Coherence Models for Dialogue

Cited by 23 publications

References 27 publications

Dialogue Coherence Assessment Without Explicit Dialogue Act Labels

Dialogue Coherence Assessment Without Explicit Dialogue Act Labels

Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators

Dialogue-Act Prediction of Future Responses Based on Conversation History

Contact Info

Product

Resources

About