On Generating Extended Summaries of Long Documents

Sotudeh, Sajad; Cohan, Arman; Goharian, Nazli

doi:10.48550/arxiv.2012.14136

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While work on summarizing novels is sparse, there has been plenty of work on summarizing other kinds of long documents, such as scientific papers (Abu-Jbara and Radev, 2011;Collins et al, 2017;Subramanian et al, 2019;Cohan et al, 2018;Xiao and Carenini, 2019;Zhao et al, 2020;Sotudeh et al, 2020), and patents (Sharma et al, 2019), as well as multi-document summarization (Liu et al, 2018;Ma et al, 2020;Gharebagh et al, 2020;Chandrasekaran et al, 2020;Liu and Lapata, 2019a;Gao et al, 2020). Many of these techniques use a hierarchical approach to generating final summaries, either by having a hierarchical encoder (Cohan et al, 2018;Zhang et al, 2019c;Liu and Lapata, 2019a), or by first running an extractive summarization model followed by an abstractive model (Subramanian et al, 2019;Liu et al, 2018;Zhao et al, 2020;Gharebagh et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Recursively Summarizing Books with Human Feedback

Wu,

Ouyang,

Ziegler

et al. 2021

Preprint

View full text Add to dashboard Cite

A major challenge for scaling machine learning is training models to perform tasks that are very difficult or time-consuming for humans to evaluate. We present progress on this problem on the task of abstractive summarization of entire fiction novels. Our method combines learning from human feedback with recursive task decomposition: we use models trained on smaller parts of the task to assist humans in giving feedback on the broader task. We collect a large volume of demonstrations and comparisons from human labelers, and fine-tune GPT-3 using behavioral cloning and reward modeling to do summarization recursively. At inference time, the model first summarizes small sections of the book and then recursively summarizes these summaries to produce a summary of the entire book. Our human labelers are able to supervise and evaluate the models quickly, despite not having read the entire books themselves. Our resulting model generates sensible summaries of entire books, even matching the quality of human-written summaries in a few cases (∼ 5% of books). We achieve state-of-the-art results on the recent BookSum dataset for book-length summarization. A zero-shot question-answering model using these summaries achieves competitive results on the challenging NarrativeQA benchmark for answering questions about books and movie scripts. We release datasets of samples from our model. 2 * This was a joint project of the OpenAI Alignment team. JW and LO contributed equally. DMZ, NS, and RL were full-time contributors for most of the duration. JL and PC managed the team.

show abstract

Section: Related Workmentioning

confidence: 99%

Recursively Summarizing Books with Human Feedback

Wu,

Ouyang,

Ziegler

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The dot product of the embeddings of the two terms qi and dj may thus be used to identify their similarities. Many unrelated words have dot products that are higher than zero, indicating syntactic and semantic relatedness [18], although this is not always the case. The noise they contribute is detrimental to the retrieval model's performance.…”

Section: Amentioning

confidence: 99%

“…To uncover the relationship between words, word embedding models are trained on a huge corpus. Word embeddings are subjected to a post-processing phase as "retrofitting" [18] in this approach.…”

Section: Table 1: Similarities Between Pairs Of Wordsmentioning

confidence: 99%

Data Mining Oriented Automatic Scientific Documents Summarization

Kalyani¹,

Wankhede²,

Shahanaz³

2023

IJRITCC

View full text Add to dashboard Cite

The scientific research process usually begins with an examination of the advanced, which may include voluminous publications. Summarizing scientific articles can assist researchers in their research by speeding up the research process. The summary of scientific articles differs from the abstract text in general due to its specific structure and the inclusion of cited sentences. Most of the important information in scientific articles is presented in tables, statistics, and algorithm pseudocode. These features, however, rarely appear in the standard text. Therefore, a number of methods that consider the value of the structure of a scientific article have been suggested that improve the standard of the produced summary. This paper makes use of clustering algorithms to handle CL- SciSumm 2020 and longsumm 2020 tasks for summarization of scientific documents. There are three well-known clustering algorithms that are employed to tackle CL- SciSumm 2020 and LongSumm 2020 tasks, and several sentences recording functions, with textual deduction, are used to retrieved phrases from each cluster to generate summary.

show abstract

On Generating Extended Summaries of Long Documents

Cited by 2 publications

References 0 publications

Recursively Summarizing Books with Human Feedback

Recursively Summarizing Books with Human Feedback

Data Mining Oriented Automatic Scientific Documents Summarization

Contact Info

Product

Resources

About