EcForest: Extractive document summarization through enhanced sentence embedding and cascade forest

Abstract

We present EcForest, an extractive summarization model through Enhanced Sentence Embedding and Cascade Forest. Sentence representation is of great significance for many summarization methods. Bag-of-words mostly fails to grasp the semantics, and typical embedding models cannot capture more complex semantic features, such as polysemy and the meaning of a phrase, which is usually ignored by simply averaging the word embeddings included in a sentence. To this end, we propose Enhanced Sentence Embedding (ESE) model to solve such drawbacks via mapping several valid features to dense vectors. Essentially, the enhanced sentence embedding is a novel model for improving the distributed representation of sentence. Our sentence embedding model is universally applicable and it can be adapted to other NLP tasks. Moreover, deep forest is used as a sentence extraction algorithm for its robustness to the hyper-parameters and its efficient training algorithm compared to deep neural network.The evaluation of variant models proposed in this work proves the validation of the enhanced sentence embedding. The comparison results between EcForest and several baselines on two different datasets demonstrate that the proposed summarization model performs better than or with high competitiveness to the state-of-the-art.

A Spectral Method for Unsupervised Multi-Document Summarization

“…Specifically, the strategy utilizing tf-isf vectors has a word bag that contains all the stemmed words found in the dataset (word stemming using Porter's stemmer 5 ). For the strategy ESE, we pre-trained all different sentence embeddings on Daily Mail dataset (Hermann et al, 2015) by following the guideline of Yang et al (2019) (the dimension of concatenated embedding is 800).…”

“…Many prior works have adopted A for the MDS task, such as Yang et al (2018) and Yang et al (2019). Each element in the affinity matrix A is a pairwise affinity of two different sentences.…”

“…More details can be found in Wan et al (2007) and Wang et al (2017). ESE: the enhanced feature embedding model (Yang et al, 2019). The embedding of each sentence is the concatenation of all components: paragraph vector, positional embedding and three feature embeddings (namely word-part-of-speech, bigram and trigram).…”

Deep Learning based Text Abstraction

“…The results show that the designed approach overcomes the problem of text overload by generating effective summaries. According to K. Yang [9], EcForest is an abstraction summary model with Enhanced Sentence Embedding and Cascade Forest. Sentence representation is very important for many summarization methods.…”

“…Bags of words can barely capture semantics, and typical embedding models fail to capture more complex semantic features such as ambiguity and phrase meaning. To this end, we propose an Extended Sentence Embedding (ESE) model [9].…”