Unsupervised Feature Learning for Speech Using Correspondence and Siamese Networks

Engelbrecht, Herman A.; Kamper, Herman

doi:10.1109/lsp.2020.2973798

Cited by 19 publications

(16 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While similar to voice con-version [7,8], an explicit goal of ZeroSpeech 2019 is to learn low-bitrate representations that perform well on phone discrimination tests. In contrast to work on continuous representation learning [9][10][11][12][13], this encourages participants to find discrete units that correspond to distinct phones. 1 Early approaches to acoustic unit discovery typically combined clustering methods with hidden Markov models [15][16][17][18][19].…”

Section: Introductionmentioning

confidence: 99%

Vector-Quantized Neural Networks for Acoustic Unit Discovery in the ZeroSpeech 2020 Challenge

Niekerk¹,

Nortje²,

Kamper³

2020

Interspeech 2020

Self Cite

View full text Add to dashboard Cite

In this paper, we explore vector quantization for acoustic unit discovery. Leveraging unlabelled data, we aim to learn discrete representations of speech that separate phonetic content from speaker-specific details. We propose two neural models to tackle this challenge. Both models use vector quantization to map continuous features to a finite set of codes. The first model is a type of vector-quantized variational autoencoder (VQ-VAE). The VQ-VAE encodes speech into a discrete representation from which the audio waveform is reconstructed. Our second model combines vector quantization with contrastive predictive coding (VQ-CPC). The idea is to learn a representation of speech by predicting future acoustic units. We evaluate the models on English and Indonesian data for the ZeroSpeech 2020 challenge. In ABX phone discrimination tests, both models outperform all submissions to the 2019 and 2020 challenges, with a relative improvement of more than 30%. The discovered units also perform competitively on a downstream voice conversion task. Of the two models, VQ-CPC performs slightly better in general and is simpler and faster to train. Probing experiments show that vector quantization is an effective bottleneck, forcing the models to discard speaker information.

show abstract

Section: Introductionmentioning

confidence: 99%

Vector-Quantized Neural Networks for Acoustic Unit Discovery in the ZeroSpeech 2020 Challenge

Niekerk¹,

Nortje²,

Kamper³

2020

Interspeech 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, learning disentangled representations that reflect users' preferences can bring enhanced robustness, interpretability, and controllability. We will, in future, seek to combine different techniques like adversarial training [32] and Siamese networks [45] with disentanglement, or add further constraints grounded in information theory, to improve learning such disentangled representations from users' signals.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Privacy-preserving Voice Analysis via Disentangled Representations

Aloufi

Haddadi

Boyle

2020

Proceedings of the 2020 ACM SIGSAC Conference on Cloud Computing Security Workshop

View full text Add to dashboard Cite

Voice User Interfaces (VUIs) are increasingly popular and built into smartphones, home assistants, and Internet of Things (IoT) devices. Despite offering an always-on convenient user experience, VUIs raise new security and privacy concerns for their users. In this paper, we focus on attribute inference attacks in the speech domain, demonstrating the potential for an attacker to accurately infer a target user's sensitive and private attributes (e.g. their emotion, sex, or health status) from deep acoustic models. To defend against this class of attacks, we design, implement, and evaluate a user-configurable, privacy-aware framework for optimizing speechrelated data sharing mechanisms. Our objective is to enable primary tasks such as speech recognition and user identification, while removing sensitive attributes in the raw speech data before sharing it with a cloud service provider. We leverage disentangled representation learning to explicitly learn independent factors in the raw data. Based on a user's preferences, a supervision signal informs the filtering out of invariant factors while retaining the factors reflected in the selected preference. Our experimental evaluation over five datasets shows that the proposed framework can effectively defend against attribute inference attacks by reducing their success rates to approximately that of guessing at random, while maintaining accuracy in excess of 99% for the tasks of interest. We conclude that negotiable privacy settings enabled by disentangled representations can bring new opportunities for privacy-preserving applications.

show abstract

“…Distance between output representations for the anchor and its nearby point is trained to be minimized whilst the distance between representations for the anchor and the distant point is maximized. Both CAE and Triamese networks have also been proposed to complement one another as a joint model, referred to as Correspondence Triamese Auto-Encoder (CTAE) (Last, Engelbrecht & Kamper, 2020). CTAE learns to minimize triplet loss in the representation layer whilst using the two similar points from the triplet as input-output pair for reconstruction.…”

Section: Geometric Distance In Speech Representation Learningmentioning

confidence: 99%

Spatial position constraint for unsupervised learning of speech representations

Humayun

Yassin

Abas

2021

PeerJ Computer Science

View full text Add to dashboard Cite

The success of supervised learning techniques for automatic speech processing does not always extend to problems with limited annotated speech. Unsupervised representation learning aims at utilizing unlabelled data to learn a transformation that makes speech easily distinguishable for classification tasks, whereby deep auto-encoder variants have been most successful in finding such representations. This paper proposes a novel mechanism to incorporate geometric position of speech samples within the global structure of an unlabelled feature set. Regression to the geometric position is also added as an additional constraint for the representation learning auto-encoder. The representation learnt by the proposed model has been evaluated over a supervised classification task for limited vocabulary keyword spotting, with the proposed representation outperforming the commonly used cepstral features by about 9% in terms of classification accuracy, despite using a limited amount of labels during supervision. Furthermore, a small keyword dataset has been collected for Kadazan, an indigenous, low-resourced Southeast Asian language. Analysis for the Kadazan dataset also confirms the superiority of the proposed representation for limited annotation. The results are significant as they confirm that the proposed method can learn unsupervised speech representations effectively for classification tasks with scarce labelled data.

show abstract

Unsupervised Feature Learning for Speech Using Correspondence and Siamese Networks

Cited by 19 publications

References 35 publications

Vector-Quantized Neural Networks for Acoustic Unit Discovery in the ZeroSpeech 2020 Challenge

Vector-Quantized Neural Networks for Acoustic Unit Discovery in the ZeroSpeech 2020 Challenge

Privacy-preserving Voice Analysis via Disentangled Representations

Spatial position constraint for unsupervised learning of speech representations

Contact Info

Product

Resources

About