Towards achieving robust universal neural vocoding

Lorenzo-Trueba, Jaime; Drugman, Thomas; Latorre, Javier; Merritt, Thomas; Putrycz, Bartosz; Barra-Chicote, Roberto; Moinet, Alexis; Aggarwal, Vatsal

doi:10.48550/arxiv.1811.06292

Cited by 9 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that there is no overlap between the training data of Voxceleb1 for neural vocoder and the evaluation data of speaker verification. To further show that the vocoder adopted in the proposed method is dataset independent, we also trained a universal vocoder [41,42] with the same structure as Vocoder, but on Lrg dataset from [42], which is a large speech dataset containing 6 languages and more than 600 speakers. The vocoder trained on Lrg is denoted as "Vocoder-L".…”

Section: Griffin-lim and Parallel Waveganmentioning

confidence: 99%

Adversarial Sample Detection for Speaker Verification by Neural Vocoders

Wu¹,

Hsu²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automatic speaker verification (ASV), one of the most important technology for biometric identification, has been widely adopted in security-critical applications, including transaction authentication and access control. However, previous work has shown that ASV is seriously vulnerable to recently emerged adversarial attacks, yet effective countermeasures against them are limited. In this paper, we adopt neural vocoders to spot adversarial samples for ASV. We use the neural vocoder to re-synthesize audio and find that the difference between the ASV scores for the original and re-synthesized audio is a good indicator for discrimination between genuine and adversarial samples. This effort is, to the best of our knowledge, among the first to pursue such a technical direction for detecting adversarial samples for ASV, and hence there is a lack of established baselines for comparison. Consequently, we implement the Griffin-Lim algorithm as the detection baseline. The proposed approach achieves effective detection performance that outperforms all the baselines in all the settings. We also show that the neural vocoder adopted in the detection framework is dataset-independent. Our codes will be made open-source for future works to do fair comparison 1 .

show abstract

Section: Griffin-lim and Parallel Waveganmentioning

confidence: 99%

Adversarial Sample Detection for Speaker Verification by Neural Vocoders

Wu¹,

Hsu²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The model, which is described in [5], contains two Gated Recurrent Units (GRU) and two dense layers. We first concatenate the quantized speaker embedding to the quantized encoding and pass the resulting tensor through the first GRU.…”

Section: Autoregressive Decodermentioning

confidence: 99%

“…Recently, there have been successful efforts in building learned codecs; starting with replacing the decoders with learned decoders for fixed encoders, which can operate as low as 2.4 kb/s to 1.6 kb/s [2,3]. These learned decoders leverage advances in speech synthesizing generative models such as WaveNet, WaveRNN, and LPCNet [4,5,6]. More recently, in [7,8], the encoder and decoder were both learned in a joint fashion, by using quantized bottlenecks based on Vector-Quantized Variational Auto-Encoders (VQ-VAE) [9], and soft-to-hard quantizers.…”

Section: Introductionmentioning

confidence: 99%

Enhancing into the codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders

Casebeer¹,

Vale²,

Isik³

et al. 2021

Preprint

View full text Add to dashboard Cite

Audio codecs based on discretized neural autoencoders have recently been developed and shown to provide significantly higher compression levels for comparable quality speech output. However, these models are tightly coupled with speech content, and produce unintended outputs in noisy conditions. Based on VQ-VAE autoencoders with WaveRNN decoders, we develop compressor-enhancer encoders and accompanying decoders, and show that they operate well in noisy conditions. We also observe that a compressor-enhancer model performs better on clean speech inputs than a compressor model trained only on clean speech.

show abstract

“…First, a seq2seq acoustic model predicts mel-spectrograms from a sequence of phoneme-level linguistic inputs. Then a speaker-independent neural vocoder converts the mel-spectrograms into a highfidelity audio waveform [12].…”

Section: Baseline Modelmentioning

confidence: 99%

Fine-grained robust prosody transfer for single-speaker neural text-to-speech

Klimkov¹,

Ronanki²,

Rohnke³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We present a neural text-to-speech system for fine-grained prosody transfer from one speaker to another. Conventional approaches for end-to-end prosody transfer typically use either fixed-dimensional or variable-length prosody embedding via a secondary attention to encode the reference signal. However, when trained on a single-speaker dataset, the conventional prosody transfer systems are not robust enough to speaker variability, especially in the case of a reference signal coming from an unseen speaker. Therefore, we propose decoupling of the reference signal alignment from the overall system. For this purpose, we pre-compute phoneme-level time stamps and use them to aggregate prosodic features per phoneme, injecting them into a sequence-to-sequence text-to-speech system. We incorporate a variational auto-encoder to further enhance the latent representation of prosody embeddings. We show that our proposed approach is significantly more stable and achieves reliable prosody transplantation from an unseen speaker. We also propose a solution to the use case in which the transcription of the reference signal is absent. We evaluate all our proposed methods using both objective and subjective listening tests.

show abstract

Towards achieving robust universal neural vocoding

Cited by 9 publications

References 0 publications

Adversarial Sample Detection for Speaker Verification by Neural Vocoders

Adversarial Sample Detection for Speaker Verification by Neural Vocoders

Enhancing into the codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders

Fine-grained robust prosody transfer for single-speaker neural text-to-speech

Contact Info

Product

Resources

About