A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization

Wang, Li; Yao, Junlin; Tao, Yunzhe; Zhong, Li; Liu, Wei; Du, Qiang

doi:10.24963/ijcai.2018/619

Cited by 110 publications

(70 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By the end of the training process (where η = 1), the model completely uses the REINFORCE loss for training. This mixed training loss was used in many of the recent works on text summarization [13], [46], [157], [158], paraphrase generation [159], image captioning [40], video captioning [160], speech recognition [161], dialogue generation [162], question answering [163], and question generation [62].…”

Section: A Policy Gradient and Reinforce Algorithmmentioning

confidence: 99%

Deep Reinforcement Learning for Sequence-to-Sequence Models

Keneshloo

Shi

Ramakrishnan

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

143

View full text Add to dashboard Cite

In recent times, sequence-to-sequence (seq2seq) models have gained a lot of popularity and provide state-ofthe-art performance in a wide variety of tasks such as machine translation, headline generation, text summarization, speech to text conversion, and image caption generation. The underlying framework for all these models is usually a deep neural network comprising an encoder and a decoder. Although simple encoderdecoder models produce competitive results, many researchers have proposed additional improvements over these seq2seq models, e.g., using an attention-based model over the input, pointergeneration models, and self-attention models. However, such seq2seq models suffer from two common problems: 1) exposure bias and 2) inconsistency between train/test measurement. Recently, a completely novel point of view has emerged in addressing these two problems in seq2seq models, leveraging methods from reinforcement learning (RL). In this survey, we consider seq2seq problems from the RL point of view and provide a formulation combining the power of RL methods in decision-making with seq2seq models that enable remembering long-term memories. We present some of the most recent frameworks that combine concepts from RL and deep neural networks. Our work aims to provide insights into some of the problems that inherently arise with current approaches and how we can address them with better RL models. We also provide the source code for implementing most of the RL models discussed in this paper to support the complex task of abstractive text summarization and provide some targeted experiments for these RL models, both in terms of performance and training time.

show abstract

Section: A Policy Gradient and Reinforce Algorithmmentioning

confidence: 99%

Deep Reinforcement Learning for Sequence-to-Sequence Models

Keneshloo

Shi

Ramakrishnan

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

143

View full text Add to dashboard Cite

show abstract

“…A CNN-based architecture is recently employed by Gehring et al (2017) using ConvS2S, which applies CNN on both encoder and decoder. Later, Wang et al (2018) build upon ConvS2S with topic words embedding and encoding, and train the system with reinforcement learning.…”

Section: Related Workmentioning

confidence: 99%

Contrastive Attention Mechanism for Abstractive Sentence Summarization

Duan¹,

Yu²,

Yin³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

We propose a contrastive attention mechanism to extend the sequence-to-sequence framework for abstractive sentence summarization task, which aims to generate a brief summary of a given source sentence. The proposed contrastive attention mechanism accommodates two categories of attention: one is the conventional attention that attends to relevant parts of the source sentence, the other is the opponent attention that attends to irrelevant or less relevant parts of the source sentence. Both attentions are trained in an opposite way so that the contribution from the conventional attention is encouraged and the contribution from the opponent attention is discouraged through a novel softmax and softmin functionality. Experiments on benchmark datasets show that, the proposed contrastive attention mechanism is more focused on the relevant parts for the summary than the conventional attention mechanism, and greatly advances the stateof-the-art performance on the abstractive sentence summarization task. We release the code at https://github.com/travel-go/ Abstractive-Text-Summarization.

show abstract

“…allowed because of limitations in the space where the headline appears. The technology of automatic headline generation has the potential to contribute greatly to this domain, and the problems of news headline generation have motivated a wide range of studies (Wang et al, 2018;Chen et al, 2018;Kiyono et al, 2018;Cao et al, 2018;Wang et al, 2019). Table 1 shows sample headlines in three different lengths written by professional editors of a media company for the same news article: The length of the first headline for the digital media is restricted to 10 characters, the second to 13 charac-ters, and the third to 26 characters.…”

Section: トヨタ、エンジン車だけの車種ゼロへ 2025年ごろmentioning

confidence: 99%

A Large-Scale Multi-Length Headline Corpus for Analyzing Length-Constrained Headline Generation Model Evaluation

Hitomi

Taguchi

Tamori

et al. 2019

Proceedings of the 12th International Conference on Natural Language Generation

View full text Add to dashboard Cite

Browsing news articles on multiple devices is now possible. The lengths of news article headlines have precise upper bounds, dictated by the size of the display of the relevant device or interface. Therefore, controlling the length of headlines is essential when applying the task of headline generation to news production. However, because there is no corpus of headlines of multiple lengths for a given article, previous research on controlling output length in headline generation has not discussed whether the system outputs could be adequately evaluated without multiple references of different lengths. In this paper, we introduce two corpora, which are Japanese News Corpus (JNC) and JApanese MUlti-Length Headline Corpus (JAMUL), to confirm the validity of previous evaluation settings. The JNC provides common supervision data for headline generation. The JAMUL is a large-scale evaluation dataset for headlines of three different lengths composed by professional editors. We report new findings on these corpora; for example, although the longest length reference summary can appropriately evaluate the existing methods controlling output length, this evaluation setting has several problems.

show abstract

A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization

Cited by 110 publications

References 1 publication

Deep Reinforcement Learning for Sequence-to-Sequence Models

Deep Reinforcement Learning for Sequence-to-Sequence Models

Contrastive Attention Mechanism for Abstractive Sentence Summarization

A Large-Scale Multi-Length Headline Corpus for Analyzing Length-Constrained Headline Generation Model Evaluation

Contact Info

Product

Resources

About