Sparse representation for frequency warping based voice conversion

Tian, Xinmei; Wu, Zhizheng; Lee, Siu Wa; Hy, Nguyen Quy; Chng, Eng Siong; Dong, Minghui

doi:10.1109/icassp.2015.7178769

Cited by 18 publications

(33 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are normally two ways to handle AP in voice conversion. The first one directly uses the AP of source speaker to synthesize converted speech [17,12,18]. The second one performs model based band aperiodicity (BAP) conversion, e.g., GMM based mapping.…”

Section: Aperiodicity Conversionmentioning

confidence: 99%

The USTC System for Voice Conversion Challenge 2016: Neural Network Based Approaches for Spectrum, Aperiodicity and F0 Conversion

et al. 2016

View full text Add to dashboard Cite

This paper introduces the methods we adopt to build our system for the evaluation event of Voice Conversion Challenge (VCC) 2016. We propose to use neural network-based approaches to convert both spectral and excitation features. First, the generatively trained deep neural network (GTDNN) is adopted for spectral envelope conversion after the spectral envelopes have been pre-processed by frequency warping. Second, we propose to use a recurrent neural network (RNN) with long short-term memory (LSTM) cells for F0 trajectory conversion. In addition, we adopt a DNN for band aperiodicity conversion. Both internal tests and formal VCC evaluation results demonstrate the effectiveness of the proposed methods.

show abstract

Section: Aperiodicity Conversionmentioning

confidence: 99%

The USTC System for Voice Conversion Challenge 2016: Neural Network Based Approaches for Spectrum, Aperiodicity and F0 Conversion

et al. 2016

View full text Add to dashboard Cite

show abstract

“…FW is another technique to tackle over-smoothing [35], [10], [37], [9], [38]. By limiting the conversion to only warping the frequency axis of a high-dimensional source spectrum towards the target spectrum, spectral details are preserved.…”

Section: Introductionmentioning

confidence: 99%

High quality voice conversion using prosodic and high-resolution spectral features

Nguyen

Lee

Tian

et al. 2015

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

Voice conversion methods have advanced rapidly over the last decade. Studies have shown that speaker characteristics are captured by spectral feature as well as various prosodic features. Most existing conversion methods focus on the spectral feature as it directly represents the timbre characteristics, while some conversion methods have focused only on the prosodic feature represented by the fundamental frequency. In this paper, a comprehensive framework using deep neural networks to convert both timbre and prosodic features is proposed. The timbre feature is represented by a high-resolution spectral feature. The prosodic features include F0, intensity and duration. It is well known that DNN is useful as a tool to model highdimensional features. In this work, we show that DNN initialized by our proposed autoencoder pretraining yields good quality DNN conversion models. This pretraining is tailor-made for voice conversion and leverages on autoencoder to capture the generic spectral shape of source speech. Additionally, our framework uses segmental DNN models to capture the evolution of the prosodic features over time. To reconstruct the converted speech, the spectral feature produced by the DNN model is combined with the three prosodic features produced by the DNN segmental models. Our experimental results show that the application of both prosodic and high-resolution spectral features leads to quality converted speech as measured by objective evaluation and subjective listening tests.

show abstract

“…Data Voice Conversion [29][30][31]: In the first study, we introduce a framework based on exemplar-based frequency warping and Residual Compensation (RC) for parallel data voice conversion. In previous frequency warping framework, the GMM was adopted to generate the warping functions and spectral residuals, which introduced the over-smoothing effect and degraded system performance.…”

Section: An Exemplar-based Frequency Warping (Efw) Approach For Parallelmentioning

confidence: 99%

“…In this way, the search space of AMF is reduced. Operating on formant frequencies, AMF reduces the over-stretched or over-compressed problems caused by DFW to obtain a better warping function [30]. More recently, a Correlation-based Frequency Warping (CFW) [29] is proposed to improve the AMF, where it allows for a more relaxed alignment between the source and target around the formants.…”

Section: Warping Function Estimationmentioning

confidence: 99%

“…The first work on the exemplar-based approach to frequency warping discussed in Chapter 3 is published in the conferences: ISCSLP 2014 [29], ICASSP 2015 [30], and its extension is published in the journal: IEEE/ACM Transactions on Audio, Speech and Language Processing [31]. The second work on the average model approach discussed in Chapter 4 is published in the conference: Odyssey 2018 [32].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Voice conversion with parallel/non-parallel data and synthetic speech detection

Tian¹

Self Cite

View full text Add to dashboard Cite

These past four years have been a wonderful experience for me. I could not have completed this work successfully without the love and support from a lot of people around me. Firstly, I would like to express my sincere thanks to my supervisor Dr. Chng Eng Siong (Nanyang Technological University, NTU) and co-supervisor Dr. Lee Siu Wa (Institute for Infocomm Research, I 2 R) for their support and advice. Their suggestions and discussions helped me to overcome the difficulties in my research. I would also like to express my appreciation to Prof. Li Haizhou and Dr. Wu Zhizheng for all their guidance and collaboration. They were like co-supervisors to me, discussing research ideas and revising conference and journal papers, which are the important parts of my Ph.D research. Moreover, I also want to thank to my colleagues in Human Language Technology department of I 2 R for their generous help. Also, thanks to all my colleagues at speech team and NTU-UBC Research Center of Excellence in Active Living for the Elderly (LILY) in NTU. It is a great pleasure to meet all these friends and discuss different technical aspects of the project and research work.

show abstract

Sparse representation for frequency warping based voice conversion

Cited by 18 publications

References 18 publications

The USTC System for Voice Conversion Challenge 2016: Neural Network Based Approaches for Spectrum, Aperiodicity and F0 Conversion

The USTC System for Voice Conversion Challenge 2016: Neural Network Based Approaches for Spectrum, Aperiodicity and F0 Conversion

High quality voice conversion using prosodic and high-resolution spectral features

Voice conversion with parallel/non-parallel data and synthetic speech detection

Contact Info

Product

Resources

About