TalkSumm: A Dataset and Scalable Annotation Method for Scientific Paper Summarization Based on Conference Talks

Abstract

Currently, no large-scale training data is available for the task of scientific paper summarization. In this paper, we propose a novel method that automatically generates summaries for scientific papers, by utilizing videos of talks at scientific conferences. We hypothesize that such talks constitute a coherent and concise description of the papers' content, and can form the basis for good summaries. We collected 1716 papers and their corresponding videos, and created a dataset of paper summaries. A model trained on this dataset achieves similar performance as models trained on a dataset of summaries created manually. In addition, we validated the quality of our summaries by human experts.

TLDR: Extreme Summarization of Scientific Documents

“…Existing datasets include CSPubSum for extractive summarization (Collins et al, 2017), ArXiv and PubMed for abstract generation (Cohan et al, 2018), and SciSummNet and CL-SciSumm (Jaidka et al, 2018;Chandrasekaran et al, 2019) datasets, which incorporate citation contexts into human-written summaries. TalkSumm (Lev et al, 2019) uses recordings of conference talks to create a distantly-supervised training set for the CL-SciSumm task.…”

“…We use the OpenReview API 5 to collect pairs of papers and author-written TLDRs, along with the (Jaidka et al, 2018;Chandrasekaran et al, 2019), which has an additional 40 manually annotated documents and its statistics are similar to SciSummNet. ‡ Unlike the other summarization datasets presented here, TalkSumm is an automatically-constructed dataset for training; the TalkSumm-supervised model in Lev et al (2019) was evaluated using CL-SciSumm (Jaidka et al, 2018).…”

CIST@CL-SciSumm 2020, LongSumm 2020: Automatic Scientific Document Summarization

“…The abstractive summarization data are from published papers and blogs which contain around 700 articles with an average of 31.7 sentences per summary and an average of 21.6 words per sentence. The extractive data are from Lev et al (2019) which have 1705 paper-summary pairs. For each paper, it provides a summary with 30 sentences and 990 words on average.…”

IIITBH-IITP@CL-SciSumm20, CL-LaySumm20, LongSumm20

“…The training corpus for this task includes 1705 extractive summaries, and 531 abstractive summaries of NLP/ML scientific papers. The extractive summaries are based on video talks from associated conferences (Lev et al, 2019), while the abstractive summaries are from blog posts created by NLP and ML researchers. The test set consists of 22 research papers for both extactive and abstractive summarization and task is to generate a summary of 600 words.…”