Multi-task WaveNet: A Multi-task Generative Model for Statistical Parametric Speech Synthesis without Fundamental Frequency Conditions

Abstract

This paper introduces an improved generative model for statistical parametric speech synthesis (SPSS) based on WaveNet under a multi-task learning framework. Different from the original WaveNet model, the proposed Multi-task WaveNet employs the frame-level acoustic feature prediction as the secondary task and the external fundamental frequency prediction model for the original WaveNet can be removed. Therefore the improved WaveNet can generate high-quality speech waveforms only conditioned on linguistic features. Multi-task WaveNet can produce more natural and expressive speech by addressing the pitch prediction error accumulation issue and possesses more succinct inference procedures than the original WaveNet. Experimental results prove that the SPSS method proposed in this paper can achieve better performance than the state-of-the-art approach utilizing the original WaveNet in both objective and subjective preference tests.

DeepFake Detection: Current Challenges and Next Steps

“…Audio DeepFakes. AI-based impersonation are not limited to imagery, recent AI-synthesized content-generation are leading to the creation of highly realistic audios [28,29]. Using synthesized audios of the impersonating target can significantly make the DeepFake videos more convincing and compounds its negative impact.…”

Prosodic Prominence and Boundaries in Sequence-to-Sequence Speech Synthesis

“…Although objective measures do not directly correlate with subjective measures of human perception, they provide the means to assess the overall model performance (see, e.g. [21,22]). The objective measures used in the current setup are the root-mean-squared error (RMSE) and Pearson correlation between the reference and the synthesized signal in terms of (i) the f0 over the voiced intervals, (ii) the voiced energy, (iii) phone duration, and (iv) word duration.…”

Detection of AI-Synthesized Speech Using Cepstral & Bispectral Statistics

“…In today's era, advances in Artificial Intelligence and Deep Neural Networks have led to very significant results in creating a more realistic type of synthesized audio and speech [2], [4]. Speech cloning and duplication via training the neural networks using powerful AI algorithms lead to synthesized speech.…”