STRASS: A Light and Effective Method for Extractive Summarization Based on Sentence Embeddings

Bouscarrat, Léo; Bonnefoy, Antoine; Peel, Thomas; Pereira, Cécile

doi:10.18653/v1/p19-2034

Cited by 14 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…M7 -STRASS (Bouscarrat et al, 2019) extracts a summary by selecting the sentences with the closest embeddings to the document embedding, learning a transformation to maximize the similarity between the summary and the ground truth reference. See et al (2017) propose a variation of encoder-decoder models, the Pointer Generator Network, where the decoder can choose to generate a word from the vocabulary or copy a word from the input.…”

Section: M1mentioning

confidence: 99%

SummEval: Re-evaluating Summarization Evaluation

Fabbri

Kryściński

McCann

et al. 2021

Transactions of the Association for Computational Linguistics

265

166

View full text Add to dashboard Cite

The scarcity of comprehensive up-to-date studies on evaluation metrics for text summarization and the lack of consensus regarding evaluation protocols continue to inhibit progress. We address the existing shortcomings of summarization evaluation methods along five dimensions: 1) we re-evaluate 14 automatic evaluation metrics in a comprehensive and consistent fashion using neural summarization model outputs along with expert and crowd-sourced human annotations; 2) we consistently benchmark 23 recent summarization models using the aforementioned automatic evaluation metrics; 3) we assemble the largest collection of summaries generated by models trained on the CNN/DailyMail news dataset and share it in a unified format; 4) we implement and share a toolkit that provides an extensible and unified API for evaluating summarization models across a broad range of automatic metrics; and 5) we assemble and share the largest and most diverse, in terms of model types, collection of human judgments of model-generated summaries on the CNN/Daily Mail dataset annotated by both expert judges and crowd-source workers. We hope that this work will help promote a more complete evaluation protocol for text summarization as well as advance research in developing evaluation metrics that better correlate with human judgments.

show abstract

Section: M1mentioning

confidence: 99%

SummEval: Re-evaluating Summarization Evaluation

Fabbri

Kryściński

McCann

et al. 2021

Transactions of the Association for Computational Linguistics

265

166

View full text Add to dashboard Cite

show abstract

“…vance labels of sentences in a document as binarylatent variables M4 -REFRESH(Narayan et al, 2018) propose using REINFORCE (Williams, 1992) to extract summaries, approximating the search space during training by limiting to combinations of individually high-scoring sentences. M5 -RNES (Wu and Hu, M7 -STRASS(Bouscarrat et al, 2019) extracts a summary by selecting the sentences with the closest embeddings to the document embedding, learning a transformation to maximize the similarity between the summary and the ground truth reference. M11 -Improve-abs(Kryściński et al, 2018) extend the model ofPaulus et al (2017) by augmenting the decoder with an external LSTM language model and add a novelty RL-based objective during training.…”

mentioning

confidence: 99%

SummEval: Re-evaluating Summarization Evaluation

Fabbri¹,

Kryściński²,

McCann³

et al. 2020

Preprint

View full text Add to dashboard Cite

The scarcity of comprehensive up-to-date studies on evaluation metrics for text summarization and the lack of consensus regarding evaluation protocols continues to inhibit progress. We address the existing shortcomings of summarization evaluation methods along five dimensions: 1) we reevaluate 12 automatic evaluation metrics in a comprehensive and consistent fashion using neural summarization model outputs along with expert and crowd-sourced human annotations, 2) we consistently benchmark 23 recent summarization models using the aforementioned automatic evaluation metrics, 3) we assemble the largest collection of summaries generated by models trained on the CNN/DailyMail news dataset and share it in a unified format, 4) we implement and share a toolkit that provides an extensible and unified API for evaluating summarization models across a broad range of automatic metrics, 5) we assemble and share the largest and most diverse, in terms of model types, collection of human judgments of model-generated summaries on the CNN/Daily Mail dataset annotated by both expert judges and crowd source workers. We hope that this work will help promote a more complete evaluation protocol for text summarization as well as advance research in developing evaluation metrics that better correlate with human judgements.

show abstract

“…Paulus et al [4] propose a network with a novel intra-attention for the abstractive summarization of long documents. Bouscarrat et al [6] propose a light model based on Transformer for extractive summarization. However, most of these deep learning mechanisms are supervised models and they need a large number of training data.…”

Section: Related Workmentioning

confidence: 99%

“…Many sequence-to-sequence models [1][2][3] and attention-based models [4] are proposed for this task. BERT [5] is also used in existing deep learning models to achieve a better performance [6,7]. These studies have achieved promising results in document summarization.…”

Section: Introductionmentioning

confidence: 99%

Incorporating External Knowledge into Unsupervised Graph Model for Document Summarization

et al. 2020

View full text Add to dashboard Cite

Supervised neural network models have achieved outstanding performance in the document summarization task in recent years. However, it is hard to get enough labeled training data with a high quality for these models to generate different types of summaries in reality. In this work, we mainly focus on improving the performance of the popular unsupervised Textrank algorithm that requires no labeled training data for extractive summarization. We first modify the original edge weight of Textrank to take the relative position of sentences into account, and then combine the output of the improved Textrank with K-means clustering to improve the diversity of generated summaries. To further improve the performance of our model, we innovatively incorporate external knowledge from open-source knowledge graphs into our model by entity linking. We use the knowledge graph sentence embedding and the tf-idf embedding as the input of our improved Textrank, and get the final score for each sentence by linear combination. Evaluations on the New York Times data set show the effectiveness of our knowledge-enhanced approach. The proposed model outperforms other popular unsupervised models significantly.

show abstract

STRASS: A Light and Effective Method for Extractive Summarization Based on Sentence Embeddings

Cited by 14 publications

References 15 publications

SummEval: Re-evaluating Summarization Evaluation

SummEval: Re-evaluating Summarization Evaluation

SummEval: Re-evaluating Summarization Evaluation

Incorporating External Knowledge into Unsupervised Graph Model for Document Summarization

Contact Info

Product

Resources

About