Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies

Grusky, Max; Нааман, Мор; Artzi, Yoav

doi:10.18653/v1/n18-1065

Cited by 402 publications

(503 citation statements)

References 14 publications

Supporting

Mentioning

500

Contrasting

Unclassified

Order By: Relevance

“…CNN-DM and AnchorContext generate fluent texts with low perplexities. While CNN-DM is trained on well-written news articles with extractive summaries [13], the high performance (low perplexity) of the Anchor-Context model can be attributed to the relatively large corpus of 10 million training examples. However, in case of AnchorContext-QB, just the addition of query bias in the snippet generation process introduces breaks in the text flow, thereby introducing a small increase in the perplexities.…”

Section: Intrinsic Evaluationmentioning

confidence: 99%

Abstractive Snippet Generation

Chen

Syed

Stein

et al. 2020

Proceedings of the Web Conference 2020

View full text Add to dashboard Cite

An abstractive snippet is an originally created piece of text to summarize a web page on a search engine results page. Compared to the conventional extractive snippets, which are generated by extracting phrases and sentences verbatim from a web page, abstractive snippets circumvent copyright issues; even more interesting is the fact that they open the door for personalization. Abstractive snippets have been evaluated as equally powerful in terms of user acceptance and expressiveness-but the key question remains: Can abstractive snippets be automatically generated with sufficient quality?This paper introduces a new approach to abstractive snippet generation: We identify the first two large-scale sources for distant supervision, namely anchor contexts and web directories. By mining the entire ClueWeb09 and ClueWeb12 for anchor contexts and by utilizing the DMOZ Open Directory Project, we compile the Webis Abstractive Snippet Corpus 2020, comprising more than 3.5 million triples of the form ⟨query, snippet, document⟩ as training examples, where the snippet is either an anchor context or a web directory description in lieu of a genuine query-biased abstractive snippet of the web document. We propose a bidirectional abstractive snippet generation model and assess the quality of both our corpus and the generated abstractive snippets with standard measures, crowdsourcing, and in comparison to the state of the art. The evaluation shows that our novel data sources along with the proposed model allow for producing usable query-biased abstractive snippets while minimizing text reuse.

show abstract

Section: Intrinsic Evaluationmentioning

confidence: 99%

Abstractive Snippet Generation

Chen

Syed

Stein

et al. 2020

Proceedings of the Web Conference 2020

View full text Add to dashboard Cite

show abstract

“…It has multiple sentences (4.0 on average) as a summary. • Newsroom (Grusky et al, 2018): contains 1.3M news articles and written summaries by authors and editors from 1998 to 2017. It has both extractive and abstractive summaries.…”

Section: Summarization Corporamentioning

confidence: 99%

Earlier Isn’t Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization

Jung¹,

Kang²,

Mentch³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Despite the recent developments on neural summarization systems, the underlying logic behind the improvements from the systems and its corpus-dependency remains largely unexplored. Position of sentences in the original text, for example, is a well known bias for news summarization. Following in the spirit of the claim that summarization is a combination of sub-functions, we define three sub-aspects of summarization: position, importance, and diversity and conduct an extensive analysis of the biases of each sub-aspect with respect to the domain of nine different summarization corpora (e.g., news, academic papers, meeting minutes, movie script, books, posts). We find that while position exhibits substantial bias in news articles, this is not the case, for example, with academic papers and meeting minutes. Furthermore, our empirical study shows that different types of summarization systems (e.g., neural-based) are composed of different degrees of the sub-aspects. Our study provides useful lessons regarding consideration of underlying sub-aspects when collecting a new summarization dataset or developing a new system.

show abstract

“…The summarization objective is to select a handful of sentences to maximize the coverage of important content while minimizing summary redundancy. Although unsupervised methods are promising, they cannot benefit from the large-scale training data harvested from the Web (Sandhaus, 2008;Hermann et al, 2015;Grusky et al, 2018). Neural extractive summarization has focused primarily on extracting sentences (Nallapati et al, 2017;Cao et al, 2017;Isonuma et al, 2017;Tarnpradab et al, 2017;Zhou et al, 2018;Kedzie et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Guiding Extractive Summarization with Question-Answering Rewards

Arumae¹,

Liu²

2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Highlighting while reading is a natural behavior for people to track salient content of a document. It would be desirable to teach an extractive summarizer to do the same. However, a major obstacle to the development of a supervised summarizer is the lack of ground-truth. Manual annotation of extraction units is costprohibitive, whereas acquiring labels by automatically aligning human abstracts and source documents can yield inferior results. In this paper we describe a novel framework to guide a supervised, extractive summarization system with question-answering rewards. We argue that quality summaries should serve as a document surrogate to answer important questions, and such question-answer pairs can be conveniently obtained from human abstracts. The system learns to promote summaries that are informative, fluent, and perform competitively on question-answering. Our results compare favorably with those reported by strong summarization baselines as evaluated by automatic metrics and human assessors.

show abstract

Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies

Cited by 402 publications

References 14 publications

Abstractive Snippet Generation

Abstractive Snippet Generation

Earlier Isn’t Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization

Guiding Extractive Summarization with Question-Answering Rewards

Contact Info

Product

Resources

About