Disentanglement by Cyclic Reconstruction

Bertoin, David; Rachelson, Emmanuel

doi:10.1109/tnnls.2022.3212620

Cited by 3 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, it is always advantageous in obtaining representations that carry certain attributes or extract discriminative features. Reconstruction based training (Gonzalez-Garcia et al, 2018;Zhang et al, 2019;Bertoin and Rachelson, 2022) is widely adopted in disentanglement learning and used to obtain disentangled representations. The application of representation disentanglement is extensive, including speech (Chan and Ghosh, 2022;), computer vision (Gonzalez-Garcia et al, 2018Lee et al, 2021) and natural language precessing (Bao et al, 2019;Cheng et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

“…Cyclic Reconstruction To effectively disentangle speech representations, we employ the cyclic reconstruction method proposed by Bertoin and Rachelson (2022) to reduce the mutual information between the content and non-content representations in an unsupervised learning setting. Specifically, after the extraction of content and non-content representations from the content and non-content encoder respectively, we stack two sub-networks: the content feature predictor ϕ content and non-content feature predictor ϕ non−content to cyclically reconstruct the content and non-content representations.…”

Section: Speech Representation Disentanglement Learningmentioning

confidence: 99%

See 1 more Smart Citation

CCSRD: Content-Centric Speech Representation Disentanglement Learning for End-to-End Speech Translation

Zhao,

Sun,

Lei

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Deep neural networks have demonstrated their capacity in extracting features from speech inputs. However, these features may include non-linguistic speech factors such as timbre and speaker identity, which are not directly related to translation. In this paper, we propose a content-centric speech representation disentanglement learning framework for speech translation, CCSRD, which decomposes speech representations into content representations and non-linguistic representations via representation disentanglement learning. CCSRD consists of a content encoder that encodes linguistic content information from the speech input, a non-content encoder that models non-linguistic speech features, and a disentanglement module that learns disentangled representations with a cyclic reconstructor, feature reconstructor and speaker classifier trained in a multi-task learning way. Experiments on the MuST-C benchmark dataset demonstrate that CCSRD achieves an average improvement of +0.9 BLEU in two settings across five translation directions over the baseline, outperforming state-of-the-art endto-end speech translation models and cascaded models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Speech Representation Disentanglement Learningmentioning

confidence: 99%

CCSRD: Content-Centric Speech Representation Disentanglement Learning for End-to-End Speech Translation

Zhao,

Sun,

Lei

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

show abstract

“…However, recent studies [54,25] show that certain data augmentation techniques may lead to a decrease in sample efficiency and even cause divergence. Other recent works improve the generalization performance by leveraging pre-trained image encoder [56] or segmenting important pixels from the test environment [2], etc. Unfortunately, most of them rely on knowledge or data from outer sources, e.g., ImageNet [9].…”

Section: Introductionmentioning

confidence: 99%

Building Chinese interdisciplinary research centers : the case of Tsinghua University

Li¹,

李蓉暉²

View full text Add to dashboard Cite

Graph machine learning has been extensively studied in both academia and industry. However, in the literature, most existing graph machine learning models are designed to conduct training with data samples in a random order, which may suffer from suboptimal performance due to ignoring the importance of different graph data samples and their training orders for the model optimization status. To tackle this critical problem, curriculum graph machine learning (Graph CL), which integrates the strength of graph machine learning and curriculum learning, arises and attracts an increasing amount of attention from the research community. Therefore, in this paper, we comprehensively overview approaches on Graph CL and present a detailed survey of recent advances in this direction. Specifically, we first discuss the key challenges of Graph CL and provide its formal problem definition. Then, we categorize and summarize existing methods into three classes based on three kinds of graph machine learning tasks, i.e., node-level, linklevel, and graph-level tasks. Finally, we share our thoughts on future research directions. To the best of our knowledge, this paper is the first survey for curriculum graph machine learning.

show abstract

Disentangled (Un)Controllable Features

Kooi,

Hoogendoorn,

François-Lavet

2023

2023 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

show abstract

Disentanglement by Cyclic Reconstruction

Cited by 3 publications

References 25 publications

CCSRD: Content-Centric Speech Representation Disentanglement Learning for End-to-End Speech Translation

CCSRD: Content-Centric Speech Representation Disentanglement Learning for End-to-End Speech Translation

Building Chinese interdisciplinary research centers : the case of Tsinghua University

Disentangled (Un)Controllable Features

Contact Info

Product

Resources

About