QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

Abstract

Meetings are a key component of human collaboration. As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed. However, it is hard to create a single short summary that covers all the content of a long meeting involving multiple people and topics. In order to satisfy the needs of different types of users, we define a new query-based multi-domain meeting summarization task, where models have to select and summarize relevant spans of meetings in response to a query, and we introduce QMSum, a new benchmark for this task. QMSum consists of 1,808 query-summary pairs over 232 meetings in multiple domains. Besides, we investigate a locate-then-summarize method and evaluate a set of strong summarization baselines on the task. Experimental results and manual analysis reveal that QMSum presents significant challenges in long meeting summarization for future research. Dataset is available at https://github.com/Yale-LILY/ QMSum.

SummerTime: Text Summarization Toolkit for Non-experts

“…With the advancement of neural networks (Bahdanau et al, 2014;Sutskever et al, 2014), the task of abstractive summarization has been receiving more attention (Rush et al, 2015;Chopra et al, 2016;Nallapati et al, 2016;Celikyilmaz et al, 2018;Chen and Bansal, 2018; while neural-based methods have also been developed for extractive summarization (Zhong et al, 2019b,a;Xu and Durrett, 2019;Cho et al, 2019;Zhong et al, 2020;Jia et al, 2020). Moreover, the field of text summarization has also been broadening into several subcategories, such as multi-document summarization (McKeown and Radev, 1995;Carbonell and Goldstein, 1998;Ganesan et al, 2010;, query-based summarization (Daumé III and Marcu, 2006;Otterbacher et al, 2009;Wang et al, 2016;Litvak and Vanetik, 2017;Nema et al, 2017;Baumel et al, 2018;Kulkarni et al, 2020) and dialogue summarization (Zhong et al, 2021;Chen et al, 2021a,b;Gliwa et al, 2019;Chen and Yang, 2020;. The proposed tasks, along with the datasets can also be classified by domain, such as news (Hermann et al, 2015;Narayan et al, 2018), meetings (Zhong et al, 2021;Carletta et al, 2005;Janin et al, 2003), scientifc literature (Cohan et al, 2018;Yasunaga et al, 2019), and medical records (DeYoung et al, 2021;Portet et al, 2009).…”

“…Moreover, the field of text summarization has also been broadening into several subcategories, such as multi-document summarization (McKeown and Radev, 1995;Carbonell and Goldstein, 1998;Ganesan et al, 2010;, query-based summarization (Daumé III and Marcu, 2006;Otterbacher et al, 2009;Wang et al, 2016;Litvak and Vanetik, 2017;Nema et al, 2017;Baumel et al, 2018;Kulkarni et al, 2020) and dialogue summarization (Zhong et al, 2021;Chen et al, 2021a,b;Gliwa et al, 2019;Chen and Yang, 2020;. The proposed tasks, along with the datasets can also be classified by domain, such as news (Hermann et al, 2015;Narayan et al, 2018), meetings (Zhong et al, 2021;Carletta et al, 2005;Janin et al, 2003), scientifc literature (Cohan et al, 2018;Yasunaga et al, 2019), and medical records (DeYoung et al, 2021;Portet et al, 2009).…”

“…It contains over 1,000 papers in the ACL anthology network as well as their citation networks and their manually labeled summaries. QMSum (Zhong et al, 2021) is designed for querybased multi-domain meeting summarization. It collects the meetings from AMI and ICSI dataset, as well as the committee meetings of the Welsh Parliament and Parliament of Canada.…”

“…The goal of text summarization is to generate short and fluent summaries from longer textual sources, while preserving the most salient information in them. Benefiting from recent advances of deep neural networks, in particular sequence to sequence models, with or without attention (Sutskever et al, 2014;Bahdanau et al, 2014;Vaswani et al, 2017), current state-of-the-art summarization models produce high quality summaries that can be useful in practice cases (Zhang et al, 2020a; has broadened its scope with the introduction of more summarization tasks, such as query-based summarization (Dang, 2005;Zhong et al, 2021), long-document summarization (Cohan et al, 2018), multi-document summarization (Ganesan et al, 2010;, dialogue summarization (Gliwa et al, 2019;Zhong et al, 2021). Such summarization tasks can also be from different domains (Hermann et al, 2015;Cohan et al, 2018).…”

Are We Summarizing the Right Way? A Survey of Dialogue Summarization Data Sets

“…More recently, Zhong et al (2021) used queries to represent information need when accessing the ICSI and AMI corpora.…”