Deep Communicating Agents for Abstractive Summarization

Çelikyılmaz, Aslı; Bosselut, Antoine; He, Xiaodong; Choi, Yejin

doi:10.18653/v1/n18-1150

Cited by 295 publications

(259 citation statements)

References 27 publications

Supporting

Mentioning

254

Contrasting

Order By: Relevance

“…Modern abstractive summarizers excel at finding and extracting salient content (See et al, 2017;Chen and Bansal, 2018;Celikyilmaz et al, 2018;Liu and Lapata, 2019). However, one of the key tenets of summarization is consolidation of information, and these systems can struggle to combine content from multiple source texts, yielding output summaries that contain poor grammar and even incorrect facts.…”

Section: Introductionmentioning

confidence: 99%

Analyzing Sentence Fusion in Abstractive Summarization

Lebanoff¹,

Muchovej²,

Dernoncourt³

et al. 2019

Proceedings of the 2nd Workshop on New Frontiers in Summarization

View full text Add to dashboard Cite

While recent work in abstractive summarization has resulted in higher scores in automatic metrics, there is little understanding on how these systems combine information taken from multiple document sentences. In this paper, we analyze the outputs of five state-of-the-art abstractive summarizers, focusing on summary sentences that are formed by sentence fusion. We ask assessors to judge the grammaticality, faithfulness, and method of fusion for summary sentences. Our analysis reveals that system sentences are mostly grammatical, but often fail to remain faithful to the original article.

show abstract

Section: Introductionmentioning

confidence: 99%

Analyzing Sentence Fusion in Abstractive Summarization

Lebanoff¹,

Muchovej²,

Dernoncourt³

et al. 2019

Proceedings of the 2nd Workshop on New Frontiers in Summarization

View full text Add to dashboard Cite

show abstract

“…FastAbs (Chen and Bansal, 2018) regards ROUGE scores as reward signals with reinforcement learning, which brings a great performance gain. DCA (Celikyilmaz et al, 2018) proposes deep communicating agents with reinforcement setting and achieves the best results on CNN/Daily Mail. Although our experimental results have not outperformed the state-of-the-art models, our model has a much simpler structure with fewer parameters.…”

Section: Automatic Evaluation Resultsmentioning

confidence: 99%

“…Abstractive document summarization (Rush et al, 2015;Nallapati et al, 2016;Tan et al, 2017;Chen and Bansal, 2018;Celikyilmaz et al, 2018) attempts to produce a condensed representation of the most salient information of the document, aspects of which may not appear as parts of the original input text. One popular framework used in abstractive summarization is the sequence-tosequence model introduced by Sutskever et al (2014).…”

Section: Introductionmentioning

confidence: 99%

Attention Optimization for Abstractive Document Summarization

Gui

Tian

Wang

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Attention plays a key role in the improvement of sequence-to-sequence-based document summarization models. To obtain a powerful attention helping with reproducing the most salient information and avoiding repetitions, we augment the vanilla attention model from both local and global aspects. We propose an attention refinement unit paired with local variance loss to impose supervision on the attention model at each decoding step, and a global variance loss to optimize the attention distributions of all decoding steps from the global perspective. The performances on the CNN/Daily Mail dataset verify the effectiveness of our methods.

show abstract

“…Our model improves Sentence Rewriting with BERT as an extractor and summary-level rewards to optimize the extractor. Reinforcement learning has been shown to be effective to directly optimize a non-differentiable objective in language generation including text summarization (Ranzato et al, 2016;Bahdanau et al, 2017;Paulus et al, 2018;Celikyilmaz et al, 2018;Narayan et al, 2018). Bahdanau et al (2017) use actor-critic methods for language generation, using reward shaping (Ng et al, 1999) to solve the sparsity of training signals.…”

Section: Duc-2002mentioning

confidence: 99%

Summary Level Training of Sentence Rewriting for Abstractive Summarization

Bae¹,

Kim²,

Kim³

et al. 2019

Proceedings of the 2nd Workshop on New Frontiers in Summarization

View full text Add to dashboard Cite

As an attempt to combine extractive and abstractive summarization, Sentence Rewriting models adopt the strategy of extracting salient sentences from a document first and then paraphrasing the selected ones to generate a summary. However, the existing models in this framework mostly rely on sentence-level rewards or suboptimal labels, causing a mismatch between a training objective and evaluation metric. In this paper, we present a novel training signal that directly maximizes summary-level ROUGE scores through reinforcement learning. In addition, we incorporate BERT into our model, making good use of its ability on natural language understanding. In extensive experiments, we show that a combination of our proposed model and training procedure obtains new state-of-the-art performance on both CNN/Daily Mail and New York Times datasets. We also demonstrate that it generalizes better on DUC-2002 test set.

show abstract

Deep Communicating Agents for Abstractive Summarization

Cited by 295 publications

References 27 publications

Analyzing Sentence Fusion in Abstractive Summarization

Analyzing Sentence Fusion in Abstractive Summarization

Attention Optimization for Abstractive Document Summarization

Summary Level Training of Sentence Rewriting for Abstractive Summarization

Contact Info

Product

Resources

About