Style Transfer from Non-Parallel Text by Cross-Alignment

Shen, Tianxiao; Leí, Tao; Barzilay, Regina; Jaakkola, Tommi S.

doi:10.48550/arxiv.1705.09655

Cited by 23 publications

(26 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar work to ours is the style transfer method with non-parallel text by Shen et al (2017). The authors consider a sequence-to-sequence model, where the latent state given to the decoder is also fed to a discriminator.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Machine Translation Using Monolingual Corpora Only

Lample¹,

Conneau²,

Denoyer³

et al. 2017

Preprint

120

View full text Add to dashboard Cite

Machine translation has recently achieved impressive performance thanks to recent advances in deep learning and the availability of large-scale parallel corpora. There have been numerous attempts to extend these successes to low-resource language pairs, yet requiring tens of thousands of parallel sentences. In this work, we take this research direction to the extreme and investigate whether it is possible to learn to translate even without any parallel data. We propose a model that takes sentences from monolingual corpora in two different languages and maps them into the same latent space. By learning to reconstruct in both languages from this shared feature space, the model effectively learns to translate without using any labeled data. We demonstrate our model on two widely used datasets and two language pairs, reporting BLEU scores of 32.8 and 15.1 on the Multi30k and WMT English-French datasets, without using even a single parallel sentence at training time.

show abstract

Section: Related Workmentioning

confidence: 99%

Unsupervised Machine Translation Using Monolingual Corpora Only

Lample¹,

Conneau²,

Denoyer³

et al. 2017

Preprint

120

View full text Add to dashboard Cite

show abstract

“…The discriminator is typically designed as a parameterized classifier [Tzeng et al, 2017, Goodfellow et al, 2014, Shen et al, 2017, and trained through a mini-max game with the encoder T * . However, we found this difficult to train.…”

Section: Evaluating Robustness To Attacksmentioning

confidence: 99%

NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training

Yala¹,

Esfahanizadeh²,

Oliveira³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Balancing the needs of data privacy and predictive utility is a central challenge for machine learning in healthcare. In particular, privacy concerns have led to a dearth of public datasets, complicated the construction of multi-hospital cohorts and limited the utilization of external machine learning resources. To remedy this, new methods are required to enable data owners, such as hospitals, to share their datasets publicly, while preserving both patient privacy and modeling utility. We propose NeuraCrypt 2 , a private encoding scheme based on random deep neural networks. NeuraCrypt encodes raw patient data using a randomly constructed neural network known only to the data-owner, and publishes both the encoded data and associated labels publicly. From a theoretical perspective, we demonstrate that sampling from a sufficiently rich family of encoding functions offers a welldefined and meaningful notion of privacy against a computationally unbounded adversary with full knowledge of the underlying data-distribution. We propose to approximate this family of encoding functions through random deep neural networks. Empirically, we demonstrate the robustness of our encoding to a suite of adversarial attacks and show that NeuraCrypt achieves competitive accuracy to non-private baselines on a variety of x-ray tasks. Moreover, we demonstrate that multiple hospitals, using independent private encoders, can collaborate to train improved x-ray models. Finally, we release a challenge dataset 3 to encourage the development of new attacks on NeuraCrypt. * Equal contribution 2 github.com/yala/NeuraCrypt 3 github.com/yala/NeuraCrypt-Challenge Preprint. Under review.

show abstract

“…The text style transfer task aims to change the stylistic properties (e.g., sentiment) of the text while retaining the style-independent content within the context. In particular, our work is closely related to the non-parallel text attribute transfer task [20,21]. The mainstream strategy is to formulate the style transfer problem into the "encoder-decoder" framework by explicitly disentangling the content and style in the latent space and then combining the content with a target style to achieve a transfer [20,22,23].…”

Section: Related Workmentioning

confidence: 99%

“…In particular, our work is closely related to the non-parallel text attribute transfer task [20,21]. The mainstream strategy is to formulate the style transfer problem into the "encoder-decoder" framework by explicitly disentangling the content and style in the latent space and then combining the content with a target style to achieve a transfer [20,22,23]. Most solutions adopt an adversarial paradigm to learn latent embeddings agnostic to the original style of input sentences.…”

Section: Related Workmentioning

confidence: 99%

Molecular Attributes Transfer from Non-Parallel Data

Zheng¹,

Song²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Optimizing chemical molecules for desired properties lies at the core of drug development. Despite initial successes made by deep generative models and reinforcement learning methods, these methods were mostly limited by the requirement of predefined attribute functions or parallel data with manually pre-compiled pairs of original and optimized molecules. In this paper, for the first time, we formulate molecular optimization as a style transfer problem and present a novel generative model that could automatically learn internal differences between two groups of non-parallel data through adversarial training strategies. Our model further enables both preservation of molecular contents and optimization of molecular properties through combining auxiliary guided-variational autoencoders and generative flow techniques. Experiments on two molecular optimization tasks, toxicity modification and synthesizability improvement, demonstrate that our model significantly outperforms several state-of-the-art methods.

show abstract

Style Transfer from Non-Parallel Text by Cross-Alignment

Cited by 23 publications

References 19 publications

Unsupervised Machine Translation Using Monolingual Corpora Only

Unsupervised Machine Translation Using Monolingual Corpora Only

NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training

Molecular Attributes Transfer from Non-Parallel Data

Contact Info

Product

Resources

About