WSL-DS: Weakly Supervised Learning with Distant Supervision for Query Focused Multi-Document Abstractive Summarization

Abstract

In the Query Focused Multi-Document Summarization (QF-MDS) task, a set of documents and a query are given where the goal is to generate a summary from these documents based on the given query. However, one major challenge for this task is the lack of availability of labeled training datasets. To overcome this issue, in this paper, we propose a novel weakly supervised learning approach via utilizing distant supervision. In particular, we use datasets similar to the target dataset as the training data where we leverage pre-trained sentence similarity models to generate the weak reference summary of each individual document in a document set from the multidocument gold reference summaries. Then, we iteratively train our summarization model on each single-document to alleviate the computational complexity issue that occurs while training neural summarization models in multiple documents (i.e., long sequences) at once. Experimental results on the Document Understanding Conferences (DUC) datasets show that our proposed approach sets a new state-of-the-art result in terms of various evaluation metrics.

ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summaries

“…In the future, we would like to extend ChartSumm to a multilingual dataset to address the scarcity of well-formatted datasets in other low-resource languages. We will also study how to incorporate query relevance [33][34][35], question-answering [36][37][38][39], and entity recognition [40][41][42] capabilities in this task.…”

Multihop Factual Claim Verification Using Natural Language Prompts

“…In the future, we will extend our proposed approach to a more challenging task of validating a claim using extracted evidence sentences from a retrieval system. We will also explore how domain adaptation [27] or transfer learning [28] from answer selection models [29][30][31] impacts the overall performance. Meanwhile, target word probing, where the performance of prompt-based language models is affected by the words' linguistic similarity, is also one of the potential future directions of our work.…”

Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization

“…Portions of this work have been published as short papers at the COLING 2020 (Laskar, Hoque, and Huang 2020c) and the Canadian AI 2020 (Laskar, Hoque, and Huang 2020a) conference proceedings. However, this work substantially extends the published papers in several ways, most notably: (i) we investigate the performance of different attentions in query-focused summary generation (Section 3.1); (ii) we propose a novel sequential fine-tuning approach to utilize all the available multi-document gold reference summaries for supervised training (Section 3.2); (iii) for the query-focused abstractive summarization task in single-document scenarios, we conduct several ablation tests to investigate the effectiveness of different components used in our model (Section 5.1.3) as well as case studies to analyze the effectiveness of our model in the zero-shot learning setup (Section 5.1.4), as well as summarize the key limitations of the Debatepedia dataset (Section 5.1.5); (v) for the multi-document scenario, we study how incorporating recent transformer-based pre-trained summarizers in our proposed model impact performance (Section 5.2.3); and finally, (vi) in addition to extensive experiments on benchmark datasets, we also conduct human evaluation to qualitatively compare among different models proposed for query-focused multi-document summarization (Section 5.2.4).…”