A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss

Hsu, Wan Ting; Lin, Chieh-Kai; Lee, Ming-Ying; Min, Kerui; Tang, Jing; Sun, Min

doi:10.18653/v1/p18-1013

Cited by 222 publications

(211 citation statements)

References 15 publications

Supporting

Mentioning

190

Contrasting

Order By: Relevance

“…Most content selection models train the selector with heuristic rules (Hsu et al, 2018;Li et al, 2018;Gehrmann et al, 2018;Yao et al, 2019;Moryossef et al, 2019), which fail to fully capture the relation between selection and generation. Mei et al (2016); ; ; Li et al (2018) "soft-select" word or sentence embeddings based on a gating function.…”

Section: Related Workmentioning

confidence: 99%

Select and Attend: Towards Controllable Content Selection in Text Generation

Shen¹,

Suzuki

Inui

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Many text generation tasks naturally contain two steps: content selection and surface realization. Current neural encoder-decoder models conflate both steps into a black-box architecture. As a result, the content to be described in the text cannot be explicitly controlled. This paper tackles this problem by decoupling content selection from the decoder. The decoupled content selection is human interpretable, whose value can be manually manipulated to control the content of generated text. The model can be trained end-to-end without human annotations by maximizing a lower bound of the marginal likelihood. We further propose an effective way to trade-off between performance and controllability with a single adjustable hyperparameter. In both data-to-text and headline generation tasks, our model achieves promising results, paving the way for controllable content selection in text generation.

show abstract

Section: Related Workmentioning

confidence: 99%

Select and Attend: Towards Controllable Content Selection in Text Generation

Shen¹,

Suzuki

Inui

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…With the proposed RL training procedure (BERT-ext + abs + RL), our model exceeds the best model of Chen and Bansal (2018). In addition, the result is better than those of all the other abstractive methods exploiting extractive approaches in them (Hsu et al, 2018;Chen and Bansal, 2018;Gehrmann et al, 2018).…”

Section: Cnn/daily Mailmentioning

confidence: 84%

“…More recently, there have been several studies that have attempted to improve the performance of the abstractive summarization by explicitly combining them with extractive models. Some notable examples include the use of inconsistency loss (Hsu et al, 2018), key phrase extraction (Li et al, 2018;Gehrmann et al, 2018), and sentence extraction with rewriting (Chen and Bansal, 2018). Our model improves Sentence Rewriting with BERT as an extractor and summary-level rewards to optimize the extractor.…”

Section: Duc-2002mentioning

confidence: 99%

Summary Level Training of Sentence Rewriting for Abstractive Summarization

Bae¹,

Kim²,

Kim³

et al. 2019

Proceedings of the 2nd Workshop on New Frontiers in Summarization

View full text Add to dashboard Cite

As an attempt to combine extractive and abstractive summarization, Sentence Rewriting models adopt the strategy of extracting salient sentences from a document first and then paraphrasing the selected ones to generate a summary. However, the existing models in this framework mostly rely on sentence-level rewards or suboptimal labels, causing a mismatch between a training objective and evaluation metric. In this paper, we present a novel training signal that directly maximizes summary-level ROUGE scores through reinforcement learning. In addition, we incorporate BERT into our model, making good use of its ability on natural language understanding. In extensive experiments, we show that a combination of our proposed model and training procedure obtains new state-of-the-art performance on both CNN/Daily Mail and New York Times datasets. We also demonstrate that it generalizes better on DUC-2002 test set.

show abstract

“…We adopt the widely used ROUGE [27] by pyrouge 5 for evaluation metric. It measures the similarity of the output Methods R-1 R-2 R-L Extractive Lead-3 [5] 40.34 17.70 36.57 RuNNer [14] 39.60 16.20 35.30 Refresh [17] 40.00 18.20 36.60 RNN-RL [16] 41.47 18.72 37.76 NeuSUM [18] 41.59 19.01 37.98 Abstractive Coverage [5] 39.53 17.28 36.38 Intra-attn [26] 39.87 15.82 36.90 Inconsistency [9] 40.68 17.97 37.13 Bottom-up [22] 41. 22 summary and the standard reference by computing overlapping n-gram, such as unigram, bigram and longest common subsequence (LCS).…”

Section: Resultsmentioning

confidence: 99%

“…In order to solve this problem, Point Network [21], [5] and CopyNet [4] have been proposed to allow both copying words from the original text and generating arbitrary words from a fixed vocabulary. [9] propose a unified model via inconsistency loss to combine the extractive and abstractive methods. [22] adopt bottom-up attention to alleviate the issue of Point Network tending to copy long-sequences.…”

Section: Related Workmentioning

confidence: 99%

Selective Attention Encoders by Syntactic Graph Convolutional Networks for Document Summarization

Xu¹,

Wang

Han³

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Abstractive text summarization is a challenging task, and one need to design a mechanism to effectively extract salient information from the source text and then generate a summary. A parsing process of the source text contains critical syntactic or semantic structures, which is useful to generate more accurate summary. However, modeling a parsing tree for text summarization is not trivial due to its non-linear structure and it is harder to deal with a document that includes multiple sentences and their parsing trees. In this paper, we propose to use a graph to connect the parsing trees from the sentences in a document and utilize the stacked graph convolutional networks (GCNs) to learn the syntactic representation for a document. The selective attention mechanism is used to extract salient information in semantic and structural aspect and generate an abstractive summary. We evaluate our approach on the CNN/Daily Mail text summarization dataset. The experimental results show that the proposed GCNs based selective attention approach outperforms the baselines and achieves the state-of-the-art performance on the dataset.

show abstract

A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss

Cited by 222 publications

References 15 publications

Select and Attend: Towards Controllable Content Selection in Text Generation

Select and Attend: Towards Controllable Content Selection in Text Generation

Summary Level Training of Sentence Rewriting for Abstractive Summarization

Selective Attention Encoders by Syntactic Graph Convolutional Networks for Document Summarization

Contact Info

Product

Resources

About