Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization

Abstract

The most important obstacles facing multidocument summarization include excessive redundancy in source descriptions and the looming shortage of training data. These obstacles prevent encoder-decoder models from being used directly, but optimization-based methods such as determinantal point processes (DPPs) are known to handle them well. In this paper we seek to strengthen a DPP-based method for extractive multi-document summarization by presenting a novel similarity measure inspired by capsule networks. The approach measures redundancy between a pair of sentences based on surface form and semantic information. We show that our DPP system with improved similarity measure performs competitively, outperforming strong summarization baselines on benchmark datasets. Our findings are particularly meaningful for summarizing documents created by multiple authors containing redundant yet lexically diverse expressions. 1

Multi-Document Summarization with Determinantal Point Processes and Contextualized Representations

“…We are particularly interested in leveraging BERT for better sentence quality and diversity estimates. This paper extends on previous work (Cho et al, 2019) by incorporating deep contextualized representations into DPP, with an emphasis on better sentence selection for extractive multi-document summarization. The major research contributions of this work include the following: (i) we make a first attempt to combine DPP with BERT representations to measure sentence quality and diversity and report encouraging results on benchmark summarization datasets; (ii) our findings suggest that it is best to model sentence quality, i.e., how important a sentence is to the summary, by combining semantic representations and surface indicators of the sentence, whereas pairwise sentence dissimilarity can be determined by semantic repre-sentations only; (iii) our analysis reveals that combining contextualized representations with surface features (e.g., sentence length, position, centrality, etc) remains necessary, as deep representations, albeit powerful, may not capture domain-specific semantics/knowledge such as word frequency.…”

“…Opinosis (Ganesan et al, 2010) 25.15 5.12 8.12 Extract+Rewrite (Song et al, 2018) 29.07 6.11 9.20 Pointer-Gen (See et al, 2017) 31.44 6.40 10.20 SumBasic (Vanderwende et al, 2007) 31.58 6.06 10.06 KLSumm (Haghighi et al, 2009) 31.23 7.07 10.56 LexRank (Erkan and Radev, 2004) 33.10 7.50 11.13 DPP (Kulesza and Taskar, 2012) † 36.95 9.83 13.57 DPP-Caps (Cho et al, 2019) 36.61 9.30 13.09 DPP-Caps-Comb (Cho et al, 2019) observe that DPP-BERT-Combined yields the best performance, achieving 10.23% and 11.06% Fscores respectively on DUC-04 and TAC-11. This finding suggests that sentence similarity scores and importance features from the DPP-BERT system and TF-IDF based features can complement each other to boost system performance.…”

“…Opinosis (Ganesan et al, 2010) 27.07 5.03 8.63 Extract+Rewrite (Song et al, 2018) 28.90 5.33 8.76 Pointer-Gen (See et al, 2017) 31.43 6.03 10.01 SumBasic (Vanderwende et al, 2007) 29.48 4.25 8.64 KLSumm (Haghighi et al, 2009) 31.04 6.03 10.23 LexRank (Erkan and Radev, 2004) 34.44 7.11 11.19 ICSISumm 37.31 9.36 13.12 DPP (Kulesza and Taskar, 2012) † 38.10 9.14 13.40 DPP-Caps (Cho et al, 2019) 38.25 9.22 13.40 DPP-Caps-Comb (Cho et al, 2019) representations, we concatenate u i and v i as sentence features. We also take a weighted average 2 of S ij and C ij as an estimate of pairwise sentence similarity, where C ij is the cosine similarity of sentence TF-IDF vectors.…”

“…We report ROUGE F-scores (Lin, 2004) 3 on DUC-04 (trained on DUC-03) and TAC-11 (trained on TAC-08/09/10) following standard settings (Hong et al, 2014). Ground-truth extractive summaries used in DPP training are obtained from Cho et al (2019).…”

“…). SumBasic(Vanderwende et al, 2007), KL-Sum(Haghighi and Vanderwende, 2009), and LexRank(Erkan and Radev, 2004) are extractive approaches; Opinosis (Ganesan et al, 2010), Extract+Rewrite(Song et al, 2018), and Pointer-Gen(See et al, 2017) are abstractive methods; ICSISumm is an ILP-based summarization method; and DPP-Caps-Comb, DPP-Caps are results combining DPP and capsule networks reported byCho et al (2019) w/ and w/o using sentence TF-IDF similarity (C i,j ).…”