A New Approach to Overgenerating and Scoring Abstractive Summaries

Song, Kaiqiang; Wang, Bingqing; Feng, Zhe; Liu, Fei

doi:10.18653/v1/2021.naacl-main.110

Cited by 7 publications

(6 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, we focus on auto-regressive methods in this paper. However, we believe our framework could also be applied and adopted to non auto-regressive generation models (Song et al, 2021)…”

Section: Discussionmentioning

confidence: 99%

Massive-scale Decoding for Text Generation using Lattices

Xu¹,

Jonnalagadda²,

Durrett³

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Conditional neural text generation models generate high-quality outputs, but often concentrate around a mode when what we really want is a diverse set of options. We present a search algorithm to construct lattices encoding a massive number of generation options. First, we restructure decoding as a best-first search, which explores the space differently than beam search and improves efficiency by avoiding pruning paths. Second, we revisit the idea of hypothesis recombination: we can identify pairs of similar generation candidates during search and merge them as an approximation. On both summarization and machine translation, we show that our algorithm encodes thousands of diverse options that remain grammatical and high-quality into one lattice. This algorithm provides a foundation for building downstream generation applications on top of massive-scale diverse outputs. 1

show abstract

“…Second, we focus on auto-regressive methods in this paper. However, we believe our framework could also be applied and adopted to non auto-regressive generation models (Song et al, 2021)…”

Section: Discussionmentioning

confidence: 99%

Massive-scale Decoding for Text Generation using Lattices

Xu¹,

Jonnalagadda²,

Durrett³

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…None of these studies can control the length explicitly. Song et al (2021) is able to precisely control the length by progressively filling a predetermined number of decoding slots, analogous to the vanilla NAR model in our non-autoregressive setting.…”

Section: Related Workmentioning

confidence: 99%

Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization

Liu¹,

Huang²,

Mou³

2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS first performs edit-based search towards a heuristically defined score, and generates a summary as pseudo-groundtruth. Then, we train an encoder-only non-autoregressive Transformer based on the search result. We also propose a dynamic programming approach for length-control decoding, which is important for the summarization task. Experiments on two datasets show that NAUS achieves state-of-the-art performance for unsupervised summarization, yet largely improving inference efficiency. Further, our algorithm is able to perform explicit length-transfer summary generation. 1

show abstract

“…Additionally, the need for controlling aspects of generated text such as length, style, and mentioned entity has been acknowledged lately in an effort to enable models to adapt to user input (Fan, Grangier, & Auli, 2018) or predefined user preferences (Song, Wang, Feng, & Liu, 2021). Most importantly, previous work (Sun et al, 2019;Schumann et al, 2020) pointed out how the lack of control in produced summaries led recent summarization systems to unwillingly exploit the susceptibility of automated metrics to summary length.…”

Section: Content Selection Redundancy and Length Controlmentioning

confidence: 99%

On the Trade-off between Redundancy and Local Coherence in Summarization

Cardenas¹,

Gallé²,

Cohen³

2022

Preprint

View full text Add to dashboard Cite

Extractive summarization systems are known to produce poorly coherent and, if not accounted for, highly redundant text. In this work, we tackle the problem of summary redundancy in unsupervised extractive summarization of long, highly-redundant documents. For this, we leverage a psycholinguistic theory of human reading comprehension which directly models local coherence and redundancy. Implementing this theory, our system operates at the proposition level and exploits properties of human memory representations to rank similarly content units that are coherent and non-redundant, hence encouraging the extraction of less redundant final summaries. Because of the impact of the summary length on automatic measures, we control for it by formulating content selection as an optimization problem with soft constraints in the budget of information retrieved. Using summarization of scientific articles as a case study, extensive experiments demonstrate that the proposed systems extract consistently less redundant summaries across increasing levels of document redundancy, whilst maintaining comparable performance (in terms of relevancy and local coherence) against strong unsupervised baselines according to automated evaluations.

show abstract

A New Approach to Overgenerating and Scoring Abstractive Summaries

Cited by 7 publications

References 51 publications

Massive-scale Decoding for Text Generation using Lattices

Massive-scale Decoding for Text Generation using Lattices

Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization

On the Trade-off between Redundancy and Local Coherence in Summarization

Contact Info

Product

Resources

About