How Domain Terminology Affects Meeting Summarization Performance

Abstract

Meetings are essential to modern organizations. Numerous meetings are held and recorded daily, more than can ever be comprehended. A meeting summarization system that identifies salient utterances from the transcripts to automatically generate meeting minutes can help. It empowers users to rapidly search and sift through large meeting collections. To date, the impact of domain terminology on the performance of meeting summarization remains understudied, despite that meetings are rich with domain knowledge. In this paper, we create gold-standard annotations for domain terminology on a sizable meeting corpus; they are known as jargon terms. We then analyze the performance of a meeting summarization system with and without jargon terms. Our findings reveal that domain terminology can have a substantial impact on summarization performance. We publicly release all domain terminology to advance research in meeting summarization. 1

QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

“…Most prior work in text summarization (Rush et al, 2015;Chopra et al, 2016;Nallapati et al, 2016;See et al, 2017;Celikyilmaz et al, 2018;Chen and Bansal, 2018;Zhong et al, 2019a;Xu and Durrett, 2019;Liu and Lapata, 2019;Cho et al, 2019;Jia et al, 2020) investigate how to generate better summaries on news article data, such as CNN/DailyMail (Hermann et al, 2015), Newsroom (Grusky et al, 2018), etc. Scientific paper summarization is another important branch (Cohan et al, 2018;Yasunaga et al, 2019;An et al, 2021).…”

“…Meeting summarization has attracted a lot of interest recently (Chen and Metze, 2012;Wang and Cardie, 2013;Mehdad et al, 2013;Oya et al, 2014;Shang et al, 2018;Koay et al, 2020). Specifically, Mehdad et al (2013) leverage entailment graphs and ranking strategy to generate meeting summaries.…”

A Sliding-Window Approach to Automatic Creation of Meeting Minutes

“…We observe that BART 5 We use pyrouge with default options to evaluate all summaries. The scores are different from that of Koay et al (2020) which removed stopwords during evaluation by using '-s'.…”

“…They rate each summary utterance as highly relevant (2), relevant (1) or irrelevant (0) by matching the utterance with the meeting abstract provided by the ICSI corpus. The systems for comparison are SW, TextRank and the fully supervised BERT summarizer (Koay et al, 2020). In Table 3, we report the percentage of summary utterances assigned to each category (Fleiss' Kappa=0.29).…”

“…Previous work includes efforts to extract utterances and keyphrases from meeting transcripts (Galley, 2006;Murray and Carenini, 2008;Gillick et al, 2009;, detect meeting decisions (Hsueh and Moore, 2008), compress or merge utterances to generate abstracts Wang and Cardie, 2013;Mehdad et al, 2013) and make use of acoustic-prosodic and speaker features (Maskey and Hirschberg, 2005;Zhu et al, 2009;Chen and Metze, 2012) for utterance extraction. The continued development of automatic transcription and its easy accessibility have sparked a renewed interest in meeting summarization (Shang et al, 2018;Li et al, 2019;Koay et al, 2020;Song et al, 2020;Zhu et al, 2020;Zhong et al, 2021), where neural representations are explored for this task. We believe the time is therefore ripe for a reconsideration of the approach to automatic minuting.…”