Zero-Shot Crosslingual Sentence Simplification

Abstract

Sentence simplification aims to make sentences easier to read and understand. Recent approaches have shown promising results with encoder-decoder models trained on large amounts of parallel data which often only exists in English. We propose a zero-shot modeling framework which transfers simplification knowledge from English to another language (for which no parallel simplification corpus exists) while generalizing across languages and tasks. A shared transformer encoder constructs language-agnostic representations, with a combination of task-specific encoder layers added on top (e.g., for translation and simplification). Empirical results using both human and automatic metrics show that our approach produces better simplifications than unsupervised and pivot-based methods.

Automatic Text Simplification for Social Good: Progress and Challenges

“…Due to those, it is not clear if the recently proposed neural ATS systems, published in top tier NLP/CL/AI conferences, e.g. (Nisioi et al, 2017;Zhang and Lapata, 2017;Surya et al, 2019;Kumar et al, 2020;Mallinson et al, 2020), represent a real step forward toward using ATS for social good.…”

“…The state-of-the-art ATS systems published in top tier NLP/CL/AI conferences, e.g. (Nisioi et al, 2017;Zhang and Lapata, 2017;Surya et al, 2019;Kumar et al, 2020;Mallinson et al, 2020), all describe end-to-end systems for sentence simplification, and are not directed towards any particular simplification transformation or target population.…”

“…(Coster and Kauchak, 2011a;Wubben et al, 2012;Glavaš andŠtajner, 2013Glavaš andŠtajner, , 2015Paetzold and Specia, 2016), in some a mixture of the two (Angrosh et al, 2014;Kumar et al, 2020), and in some not specified, e.g. (Specia, 2010;Xu et al, 2016;Mallinson et al, 2020;Alva-Manchego et al, 2020a). In some of those studies, the evaluators are students, i.e.…”

Klexikon: A German Dataset for Joint Summarization and Simplification

“…Arguably, unaligned resources might still be helpful to facilitate pre-training of models. In an attempt to circumvent data scarcity, (Mallinson et al, 2020) ploy multi-lingual pre-training, which they tested with a small, manually labeled German evaluation set.…”

A New Dataset and Efficient Baselines for Document-level Text Simplification in German

“…Some unsupervised or semi-supervised neural models, which reduce the need for parallel data, have reached similar performances (Surya et al, 2019;Kumar et al, 2020;Zhao et al, 2020;Martin et al, 2020b). Finally, experiments with multi-task learning have shown promising results (Guo et al, 2018;Dmitrieva and Tiedemann, 2021), with the possibility of zero-shot translations for languages without any parallel data (Mallinson et al, 2020). These approaches represent the current state of the art, but are largely limited to English (Al-Thanyyan and Azmi, 2021) due to a lack of training data in other languages.…”