Topic-Aware Contrastive Learning for Abstractive Dialogue Summarization

Abstract

Unlike well-structured text, such as news reports and encyclopedia articles, dialogue content often comes from two or more interlocutors, exchanging information with each other. In such a scenario, the topic of a conversation can vary upon progression and the key information for a certain topic is often scattered across multiple utterances of different speakers, which poses challenges to abstractly summarize dialogues. To capture the various topic information of a conversation and outline salient facts for the captured topics, this work proposes two topic-aware contrastive learning objectives, namely coherence detection and sub-summary generation objectives, which are expected to implicitly model the topic change and handle information scattering challenges for the dialogue summarization task. The proposed contrastive objectives are framed as auxiliary tasks for the primary dialogue summarization task, united via an alternative parameter updating strategy. Extensive experiments on benchmark datasets demonstrate that the proposed simple method significantly outperforms strong baselines and achieves new state-of-the-art performance. The code and trained models are publicly available via https://github.com/Junpliu/ConDigSum.

Pretrained Language Models for Text Generation: A Survey

“…Feng et al [46] utilize DialoGPT [220], a PLM specially designed for dialogue, to automatically extract the keywords, detect redundant utterances and divide a dialogue into topically coherent segments. Similarly, ConDigSum [114] detects the dialogue topic transfer and generates summaries for each topic using contrastive learning.…”

“…To address the issue of key points missing in output text, Nguyen et al [134] introduced a topic model to capture the global semantics of the document and a mechanism to control the amount of global semantics supplied to the text generation module. Similarly, Liu et al [114] also proposed two topic-aware contrastive learning objectives to capture the global topic information of a conversation and outline salient facts. These objectives are able to implicitly model the topic change varying upon conversations, pushing PLMs to focus more on snippets that contain salient information from the same topics.…”

“…These diverse auxiliary tasks can help enhance some specific aspects of the primary generation task. For example, Liu et al [114] and Jin et al [84] fine-tuned PLMs with auxiliary tasks (e.g., coherence detection, style-carrying text reconstruction) to control the content of the generated text such as the topic change and text style (humor, romance, and clickbait). Besides, to improve the faithfulness of the generated text, Li et al [103] and Gong et al [57] introduced auxiliary input reconstruction tasks to reconstruct KG triples and table values for aligning the input information and the generated content.…”

“…RL: DialogBERT[61]; Li et al[109]; Graph-based RL: BASS[191]; Ouyang et al[136] Document Encoding Inter-Sentential Semantics: BERTSumExt[119]; HIBERT et al[219]; Capturing Critical Semantics: Nguyen et al[134]; Liu et al[114]; RL Efficiency: Huang et al…”

Post-Training Dialogue Summarization using Pseudo-Paraphrasing

“…These work also modify the basic transformer-based models with additional encoders (Chen and Yang, 2020) or attention layers Lei et al, 2021; to utilize the injected features. Liu et al (2021a) propose a contrastive learning approach for dialogue summarization with multiple training objectives. They also introduce a number of hyper-parameters for contrastive dataset construction and balancing among those objectives.…”