BiSECT: Learning to Split and Rephrase Sentences with Bitexts

Abstract

An important task in NLP applications such as sentence simplification is the ability to take a long, complex sentence and split it into shorter sentences, rephrasing as necessary. We introduce a novel dataset and a new model for this 'split and rephrase' task. Our BISECT training data consists of 1 million long English sentences paired with shorter, meaning-equivalent English sentences. We obtain these by extracting 1-2 sentence alignments in bilingual parallel corpora and then using machine translation to convert both sides of the corpus into the same language. BISECT contains higher quality training examples than previous Split and Rephrase corpora, with sentence splits that require more significant modifications. We categorize examples in our corpus, and use these categories in a novel model that allows us to target specific regions of the input sentence to be split and edited. Moreover, we show that models trained on BISECT can perform a wider variety of split operations and improve upon previous state-of-the-art approaches in automatic and human evaluations. 1

Does splitting make sentence easier?

“…While there have been concerted efforts in the past in the NLP community to develop metrics and corpora purported to serve studies in simplification (Xu et al, 2015 ; Zhang and Lapata, 2017 ; Narayan et al, 2017 ; Botha et al, 2018 ; Sulem et al, 2018a ; Niklaus et al, 2019 ; Kim et al, 2021 ), they fell far short of addressing how their study contributes to improving the text comprehensibility. 3 A part of our goal is to break away from a prevailing view that relegates readability to a sideline.…”

Multilingual Simplification of Medical Texts

“…Although the CRF model handles unaligned sentences by default, the resulting data remains noisy. We therefore adapt the method described in Kim et al (2021), initially used in the context of sentence-splitting, to further filter misalignments in MC-NOISY. Ultimately, this method was applied to the entire training set, which includes the entirety of MC-NOISY as well as the portion of MC-CLEAN not used for testing or validation.…”

The Fewer Splits are Better: Deconstructing Readability in Sentence Splitting

“…While there have been concerted efforts in the past in the NLP community to develop metrics and corpora purported to serve studies in simplification (Zhang and Lapata, 2017;Sulem et al, 2018a;Botha et al, 2018;Niklaus et al, 2019;Kim et al, 2021;Xu et al, 2015), they fell far short of addressing how their work contributes to improving the text comprehensibility by readers. Part of our goal is to break away from a prevailing view that relegates the readability to a sideline.…”