MLSUM: The Multilingual Summarization Corpus

Scialom, Thomas; Dray, Paul-Alexis; Lamprier, Sylvain; Piwowarski, Benjamin; Staiano, Jacopo

doi:10.18653/v1/2020.emnlp-main.647

Cited by 98 publications

(83 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MLSUM is the first large-scale MultiLingual SUMmarization dataset which contains 1.5M+ article/summary pairs including Turkish (Scialom et al, 2020). The authors compiled the dataset following the same methodology of CNN/DailyMail dataset.…”

Section: Datasetmentioning

confidence: 99%

“…The data was split into train, validation and test sets, with respect to the publication dates. The data from 2010 to 2018 was used for training; data between January-April 2019 was used for validation; and data up to December 2019 was used for test (Scialom et al, 2020). In this study, we obtained the Turkish dataset from HuggingFace collection.…”

Section: Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Similarity Based Evaluation for Abstractive News Summarization

Fikri¹,

Oflazer²,

Yanıkoğlu³

2021

Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)

View full text Add to dashboard Cite

ROUGE is a widely used evaluation metric in text summarization. However, it is not suitable for the evaluation of abstractive summarization systems as it relies on lexical overlap between the gold standard and the generated summaries. This limitation becomes more apparent for agglutinative languages with very large vocabularies and high type/token ratios. In this paper, we present semantic similarity models for Turkish and apply them as evaluation metrics for an abstractive summarization task. To achieve this, we translated the English STSb dataset into Turkish and presented the first semantic textual similarity dataset for Turkish. We showed that our best similarity models have better alignment with average human judgments compared to ROUGE in both Pearson and Spearman correlations.

show abstract

Section: Datasetmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

Semantic Similarity Based Evaluation for Abstractive News Summarization

Fikri¹,

Oflazer²,

Yanıkoğlu³

2021

Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)

View full text Add to dashboard Cite

show abstract

“…With the large success brought by pre-trained language models in English abstractive summarization (Liu and Lapata, 2019;Lewis et al, 2020b;Zhang et al, 2020), several works focus on summarization in multiple languages. Nguyen and Daumé III (2019) constructs a small cross-lingual dataset with English summaries for non-English articles, and Scialom et al (2020) proposes MLSUM with 5 languages as the extended version of English summarization dataset CNN/DailyMail (Hermann et al, 2015). Cao et al (2020) use a Transformerbased model with 6 layers encoder and decoder to combine auto-encoder training, translation and summarization.…”

Section: Related Workmentioning

confidence: 99%

“…It lacks the ability to align sentence-level information among languages and to distinguish which information is the most critical for the document-level input. Most previous multilingual summarization models focus on training one model for different language or partly share encoder/decoder layers (Wang et al, 2018;Lin et al, 2018;Scialom et al, 2020). Cao et al (2020) and Lewis et al (2020a) try to train one model for all languages, but they find that although low-resource languages can benefit from the larger training data, the performance of rich-resource languages has been sacrificed.…”

Section: Introductionmentioning

confidence: 99%

Contrastive Aligned Joint Learning for Multilingual Summarization

Wang¹,

Chen²,

Zhou³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Multilingual text summarization requires the ability to understand documents in multiple languages and generate summaries in the corresponding language, which poses more challenges on current summarization systems. However, this problem has been rarely studied due to the lack of large-scale supervised summarization data in multiple languages. In this paper, we first provide a large-scale multilingual summarization corpus MLGSum consisting of 1.1 million articles and summaries in 12 different languages. Based on it, we develop a unified summarization model to understand the document and generate summaries in different languages. We use the contrastive learning strategy to train our multilingual summarization system (CALMS), which consists of two training objectives, contrastive sentence ranking (CSR) and sentence aligned substitution (SAS). The two training objectives are designed to share salient information extractive ability and align sentencelevel representation across different languages. Experimental results indicate that CALMS achieves significant improvement over monolingual models in all languages. We further transfer CALMS to other languages and find that it will also benefit similar languages. Our code and dataset are available at https://github.com/brxx122/CALMS.

show abstract

“…We also use MT learning in generation tasks, the tasks are extractive; i.e., the output often has significant overlap with the input. These tasks include news title generation, text summarization, and question generation (Chi et al, 2020;Liang et al, 2020;Scialom et al, 2020). Reply suggestion is more challenging because the reply often does not overlap with the message (Figure 1), so the model needs to address different cross-lingual generalization challenges (Section 5.2).…”

mentioning

confidence: 99%

A Dataset and Baselines for Multilingual Reply Suggestion

Zhang¹,

Wang²,

Deb³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Reply suggestion models help users process emails and chats faster. Previous work only studies English reply suggestion. Instead, we present MRS, a multilingual reply suggestion dataset with ten languages. MRS can be used to compare two families of models: 1) retrieval models that select the reply from a fixed set and 2) generation models that produce the reply from scratch. Therefore, MRS complements existing cross-lingual generalization benchmarks that focus on classification and sequence labeling tasks. We build a generation model and a retrieval model as baselines for MRS. The two models have different strengths in the monolingual setting, and they require different strategies to generalize across languages. MRS is publicly available at https://github.com/zhangmozhi/mrs. Multilingual Reply SuggestionAutomated reply suggestion is a useful feature for email and chat applications. Given an input message, the system suggests several replies, and users may click on them to save typing time (Figure 1). This feature is available in many applications including Gmail, Outlook, LinkedIn, Facebook Messenger, Microsoft Teams, and Uber.Reply suggestion is related to but different from open-domain dialog systems or chatbots (Adiwardana et al., 2020;. While both are conversational AI tasks (Gao et al., 2019), the goals are different: reply suggestion systems help the user quickly reply to a message, while chatbots aim to continue the conversation and focus more on multi-turn dialogues.Ideally, we want our model to generate replies in any language. However, reply suggestion models require large training sets, so previous work mostly * Work mostly done as an intern at Microsoft Research.

show abstract

MLSUM: The Multilingual Summarization Corpus

Cited by 98 publications

References 37 publications

Semantic Similarity Based Evaluation for Abstractive News Summarization

Semantic Similarity Based Evaluation for Abstractive News Summarization

Contrastive Aligned Joint Learning for Multilingual Summarization

A Dataset and Baselines for Multilingual Reply Suggestion

Contact Info

Product

Resources

About