Fact-level Extractive Summarization with Hierarchical Graph Mask on BERT

Abstract

Most current extractive summarization models generate summaries by selecting salient sentences. However, one of the problems with sentence-level extractive summarization is that there exists a gap between the human-written gold summary and the oracle sentence labels. In this paper, we propose to extract fact-level semantic units for better extractive summarization. We also introduce a hierarchical structure, which incorporates the multi-level of granularities of the textual information into the model. In addition, we incorporate our model with BERT using a hierarchical graph mask. This allows us to combine BERT's ability in natural language understanding and the structural information without increasing the scale of the model. Experiments on the CNN/DaliyMail dataset show that our model achieves state-of-the-art results.

Generating Tips from Song Reviews: A New Dataset and Framework

“…To mitigate the redundancy between extracted sentences, Zhou et al [32] extract sub-sentential units based on the constituency parsing tree to construct a summary. Wu et al [31] apply dependency analysis to sentences and propose an empirical method to split sentences into smaller units called "fact". However, these small units [28] decrease the readability of summary which is unacceptable in our task.…”

TSTR: Too Short to Represent, Summarize with Details! Intro-Guided Extended Summary Generation

“…Over the past few years, summarization task has witnessed a huge deal of progress in extractive (Nallapati et al, 2017;Liu and Lapata, 2019;Yuan et al, 2020;Cui et al, 2020;Jia et al, 2020;Feng et al, 2018) and abstractive (See et al, 2017;Gehrmann et al, 2018;Zhang et al, 2019;Tian et al, 2019;Zou et al, 2020) [Introductory] Neural machine translation (@xcite), directly applying a single neural network to transform the source sentence into the target sentence, has now reached impressive performance (@xcite […] Motivated by recent success in unsupervised cross-lingual embeddings (@xcite), the models proposed for unsupervised NMT often assume that a pair of sentences from two different languages can be mapped to a same latent representation in a shared-latent space (@xcite) […] Although the shared encoder is vital for mapping sentences from different languages into the shared-latent space, it is weak in keeping the uniqueness and internal characteristics of each language, such as the style, terminology and sentence structure. […] For each language, the encoder and its corresponding decoder perform an AE, where the encoder generates the latent representations from the perturbed input sentences and the decoder reconstructs the sentences from the latent representations.…”

TSTR: Too Short to Represent, Summarize with Details! Intro-Guided Extended Summary Generation

“…Over the past few years, summarization task has witnessed a huge deal of progress in extractive (Nallapati et al, 2017;Liu and Lapata, 2019;Yuan et al, 2020;Cui et al, 2020;Jia et al, 2020;Feng et al, 2018) and abstractive (See et al, 2017;Cohan et al, 2018;Gehrmann et al, 2018;Zhang et al, 2019;Tian et al, 2019;Zou et al, 2020) [Introductory] Neural machine translation (@xcite), directly applying a single neural network to transform the source sentence into the target sentence, has now reached impressive performance (@xcite […] Motivated by recent success in unsupervised cross-lingual embeddings (@xcite), the models proposed for unsupervised NMT often assume that a pair of sentences from two different languages can be mapped to a same latent representation in a shared-latent space (@xcite) […] Although the shared encoder is vital for mapping sentences from different languages into the shared-latent space, it is weak in keeping the uniqueness and internal characteristics of each language, such as the style, terminology and sentence structure. […] For each language, the encoder and its corresponding decoder perform an AE, where the encoder generates the latent representations from the perturbed input sentences and the decoder reconstructs the sentences from the latent representations.…”