Simple and Effective Paraphrastic Similarity from Parallel Translations

Abstract

We present a model and methodology for learning paraphrastic sentence embeddings directly from bitext, removing the timeconsuming intermediate step of creating paraphrase corpora. Further, we show that the resulting model can be applied to cross-lingual tasks where it both outperforms and is orders of magnitude faster than more complex stateof-the-art baselines. 1

Parallel Sentence Mining by Constrained Decoding

“…Guo et al (2018) design a dual encoder model to learn multilingual sentence em-beddings directly with added negative examples. Wieting et al (2019) obtain sentence embeddings from sub-word embeddings and train a simpler model to distinguish positive and negative examples. Artetxe and Schwenk (2019) refine Guo et al (2018)'s work and achieve state-of-the-art by looking at the margins of cosine similarities between pairs of nearest neighbours.…”

Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization

“…In other NLP tasks, topic information is used as conditional signals and applied to dialogue response generation (Xing et al, 2017) and pretraining of large-scale language models (Keskar et al, 2019) while sentiment polarity is used in text style transfer (John et al, 2019). In image style transfer, codes specifying color or texture are used to train conditional generative models (Mirza and Osindero, 2014;Higgins et al, 2017).…”

“…BertScore has performed better than ROUGE and BLEU in sentence-level semantic similarity assessment (Zhang et al, 2020). Moreover, BertScore includes recall measures between reference and candidate sequences, a more suitable metric than distance-based similarity measures (Wieting et al, 2019;Reimers and Gurevych, 2019) for summarization related tasks, where there is an asymmetrical relationship between the reference and the generated text.…”

“…Then, the latent vector and document vector are concatenated and fed to the decoding component to reconstruct the sentence vector. To obtain a compact yet effective latent vector representing the correlation between the source and summary, we adopt an adversarial training strategy as in (John et al, 2019). More specifically, the adversarial decoder we include aims to reconstruct the sentence vector directly from the latent vector.…”

“…Sub-Aspect Control Codes4.1 Sub-Aspect Features in News SummarizationConditional generation often uses control codes as an auxiliary vector to adjust pre-defined style features. Classic examples include sentiment polarity in style transfer(John et al, 2019) or physical attributes (e.g. color) in image generation(Higgins …”