Improving Unsupervised Extractive Summarization with Facet-Aware Modeling

Abstract

Unsupervised extractive summarization aims to extract salient sentences from documents without labeled corpus. Existing methods are mostly graph-based by computing sentence centrality. These methods usually tend to select sentences within the same facet, however, which often leads to the facet bias problem especially when the document has multiple facets (i.e. long-document and multidocuments). To address this problem, we proposed a novel facet-aware centrality-based ranking model. We let the model pay more attention to different facets by introducing a sentence-document weight. The weight is added to the sentence centrality score. We evaluate our method on a wide range of summarization tasks that include 8 representative benchmark datasets. Experimental results show that our method consistently outperforms strong baselines especially in longand multi-document scenarios and even performs comparably to some supervised models. Extensive analyses confirm that the performance gains come from alleviating the facet bias problem.

An Empirical Survey on Long Document Summarization: Datasets, Models, and Metrics

“…As there are many diferent ways to (a) encode or vectorize a sentence before calculating the similarity between them and (b) calculate the centrality score of each sentence, research involving this architecture often difers only in these two mechanisms. For example, with respect to the former mechanism, graph architecture in the past [29,83] encodes sentences based on word-occurrence or term frequency-inverse document frequency (Tf-Idf) while graph architecture today [69,135] encodes sentences with state-of-the-art pre-trained models. On the other hand, to improve the centrality scoring mechanism, PacSum [135] and FAR [69] adjust the centrality score of a sentence based on whether the other sentences come before or after it, while HipoRank [25] exploits the discourse structure contained in by adjusting the centrality score with positional and sectional bias.…”

“…For example, with respect to the former mechanism, graph architecture in the past [29,83] encodes sentences based on word-occurrence or term frequency-inverse document frequency (Tf-Idf) while graph architecture today [69,135] encodes sentences with state-of-the-art pre-trained models. On the other hand, to improve the centrality scoring mechanism, PacSum [135] and FAR [69] adjust the centrality score of a sentence based on whether the other sentences come before or after it, while HipoRank [25] exploits the discourse structure contained in by adjusting the centrality score with positional and sectional bias. In general form, given a set of sentences in the original source document, D = {s 1 , s 2 , ..., s m } with the inter-sentential similarity relations represented as e i j = (s i , s j ) ∈ E where i j, the following illustrates the aforementioned architecture in computing the scoring for each sentence:…”

Unsupervised Multi-Granularity Summarization

“…Residents are injured on graph [15,23,38], centrality [29,53], point-wise mutual information [37], or sentence-level self-attention in pre-trained models [45]. Another direction is unsupervised abstractive approaches, and these studies typically employ sequence-to-sequence autoencoding method [9] with adversarial training and reinforcement learning [42].…”

Template-based Abstractive Microblog Opinion Summarization

“…Recent work has seen the emergence of larger scale datasets such as WikiSum , Multi-News (Fabbri et al, 2019), andWCEP (Gholipour Ghalandari et al, 2020) to combat data sparsity. Extractive (Wang et al, 2020b,c;Liang et al, 2021) and abstractive (Jin et al, 2020) methods have followed from these multi-document news datasets.…”