Sentence Centrality Revisited for Unsupervised Summarization

Abstract

Single document summarization has enjoyed renewed interest in recent years thanks to the popularity of neural network models and the availability of large-scale datasets. In this paper we develop an unsupervised approach arguing that it is unrealistic to expect large-scale and high-quality training data to be available or created for different types of summaries, domains, or languages. We revisit a popular graph-based ranking algorithm and modify how node (aka sentence) centrality is computed in two ways: (a) we employ BERT, a state-of-the-art neural representation learning model to better capture sentential meaning and (b) we build graphs with directed edges arguing that the contribution of any two nodes to their respective centrality is influenced by their relative position in a document. Experimental results on three news summarization datasets representative of different languages and writing styles show that our approach outperforms strong baselines by a wide margin. 1(2) Clara Eaglen, from the royal national in-

TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising

“…(2) Unsupervised extractive systems: TextRank (Mihalcea and Tarau, 2004), Lead-X. (3) Supervised abstractive and abstractive (models trained with groundtruths summaries): PACSUM (Zheng and Lapata, 2019), PGNet (See et al, 2017), REFRESH (Narayan et al, 2018) and SUMO (Liu et al, 2019b). TED is unsupervised abstractive and therefore not directly comparable with supervised baselines.…”

“…The centrality of a node (sentence) is computed by PageRank (Brin and Page, 1998) to decide whether a sentence should be included in the final summary. Zheng and Lapata (2019) advances upon TextRank by encoding sentences with BERT representation (Devlin et al, 2018) to compute pairs similarity and build graphs with directed edges decided by the relative positions of sentences.…”

Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers

“…Then PageRank (Page et al, 1999) is employed to determine the final ranking scores for sentences. Zheng and Lapata (2019) builds directed graph by utilizing BERT (Devlin et al, 2019) to compute sentence similarities. The importance score of a sentence is the weighted sum of all its out edges, where weights for edges between the current sentence and preceding sentences are negative.…”

“…Thus, leading sentences tend to obtain high scores. Unlike Zheng and Lapata (2019), sentence positions are not explicitly modeled in our model and therefore our model is less dependent on sentence positions (as shown in experiments).…”

“…For example, Tex-tRank (Mihalcea and Tarau, 2004) utilizes word co-occurrence statistics to compute similarity and then employs PageRank (Page et al, 1997) to rank sentences. Sentence similarities in (Zheng and Lapata, 2019) are measured with BERT (Devlin et al, 2019) and sentences are sorted w.r.t. their centralities in a directed graph.…”